Cloning a function in Lua

Have you ever considered whether functions in Lua are mutable or not. In Lua, objects are mutable because properties and metatables can be changed. Strings and numbers are examples of types that aren’t mutable: string library functions return new strings, numeric operators return a new numbers.

Functions and mutability

For something to be mutable it must have state that can be changed. Depending on the version of Lua, there are either two possibilities:

• Lua 5.1: The function environment and the upvalues are mutable
• Lua 5.2 and above: the upvalues are mutable

Lua 5.2 replaced the function environment with a specially named upvalue called _ENV. You can read more about this in my companion guide: Implementing setfenv in Lua.

When assigning a function to a new variable it is not copied. Just like tables, function values are actually pointers to a function.

local a = function() end
local b = a

-- these point to the same function
assert(a == b)

This is commonly confused with pass by reference. Pass by reference is slightly different. Lua uses pass by value, it’s just that some values are pointers to the same object.

Why clone a function?

A cloned function will allow you to change state without affecting other code that is holding references to the original function.

You might think that because Lua is a single threaded language you can modify the state of the function while it executes, then put it back. This would be true if Lua didn’t have coroutines.

A running function might yield at any point in execution, and in that time the function could have its state changed before the coroutine resumes.

string.dump and loadstring

The string.dump function returns a binary representation of a function as a string. By dumping a function to a string and then reloading it you've created a clone of the function:

local function say_hi()
  print("Hi!")
end

local say_hi_clone = loadstring(string.dump(say_hi))

say_hi_clone() --> Hi!

assert(say_hi ~= say_hi_clone)

This works in the previous example but it’s not entirely correct. What about upvalues? An upvalue’s reference can not be encoded into the string dump and preserved when it’s loaded again.

local message = "Hello"

local function say_message()
  print("message: " .. tostring(message))
end

local say_message_clone = loadstring(string.dump(say_message))

say_message_clone() -- message: nil

A new set of upvalues is created for the loaded function, and they all point to nil.

Preserving upvalues

Lua 5.2 and above give two ways to set upvalues on a function: debug.setupvalue and debug.upvaluejoin. As we discovered in the setfenv implementation guide, upvalues are shared among multiple functions. Changes to the values pointed to by an upvalue should reflect in all the functions that have access. For that reason debug.upvaluejoin will be used to connect the original function’s upvalues to the new function.

debug.upvaluejoin takes two pairs of function and upvalue index. Since one function is a clone of the other, the upvalue positions will be the same. It’s just a matter of iterating through all the valid upvalue indexes and joining them.

local message = "Hello"

local function say_message()
  print("message: " .. tostring(message))
end

local say_message_clone = loadstring(string.dump(say_message))


local i = 1
while true do
  -- see if i is a valid upvalue index
  local name = debug.getupvalue(say_message, i)
  if not name then
    break
  end
  -- join the clone and the original
  debug.upvaluejoin(say_message_clone, i, say_message, i)
  i = i + 1
end

-- the clone now has a functional upvalue
say_message_clone() -- message: Hello
message = "MoonScript"
say_message_clone() -- message: MoonScript

clone_function implementation

Now all that’s left is to write a generic function for cloning any function:

local function clone_function(fn)
  local dumped = string.dump(fn)
  local cloned = loadstring(dumped)
  local i = 1
  while true do
    local name = debug.getupvalue(fn, i)
    if not name then
      break
    end
    debug.upvaluejoin(cloned, i, fn, i)
    i = i + 1
  end
  return cloned
end

Handling Lua 5.1

As far as I know Lua 5.1 does not provide a way to join upvalues. LuaJIT does provite an implementation of debug.upvaluejoin though, so that may handle any Lua runtimes you run code in.

The best alternative for Lua 5.1 is to use debug.setupvalue. The result will be a function that works, but because the upvalues aren’t connected some hard to debug issues may result.