EIP 1538: Transparent Contract Standard Source

AuthorNick Mudge
Discussions-Tohttps://github.com/ethereum/EIPs/issues/1538
StatusDraft
TypeStandards Track
CategoryERC
Created2018-10-31

Simple Summary

This standard provides a contract architecture that makes upgradeable contracts flexible, unlimited in size, and transparent.

A transparent contract publicly documents the full history of all changes made to it.

All changes to a transparent contract are reported in a standard format.

Abstract

A transparent contract is a proxy contract design pattern that provides the following:

  1. A way to add, replace and remove multiple functions of a contract atomically (at the same time).
  2. Standard events to show what functions are added, replaced and removed from a contract, and why the changes are made.
  3. A standard way to query a contract to discover and retrieve information about all functions exposed by it.
  4. Solves the 24KB maximum contract size limitation, making the maximum contract size of a transparent contract practically unlimited. This standard makes the worry about contract size a thing of the past.
  5. Enables an upgradeable contract to become immutable in the future if desired.

Motivation

A fundamental benefit of Ethereum contracts is that their code is immutable, thereby acquiring trust by trustlessness. People do not have to trust others if it is not possible for a contract to be changed.

However, a fundamental problem with trustless contracts that cannot be changed is that they cannot be changed.

Bugs

Bugs and security vulnerabilities are unwittingly written into immutable contracts that ruin them.

Improvements

Immutable, trustless contracts cannot be improved, resulting in increasingly inferior contracts over time.

Contract standards evolve, new ones come out. People, groups and organizations learn over time what people want and what is better and what should be built next. Contracts that cannot be improved not only hold back the authors that create them, but everybody who uses them.

Upgradeable Contracts vs. Centralized Private Database

Why have an upgradeable contract instead of a centralized, private, mutable database? Here are some reasons:

  1. Because of the openness of storage data and verified code, it is possible to show a provable history of trustworthiness.
  2. Because of the openness, bad behavior can be spotted and reported when it happens.
  3. Independent security and domain experts can review the change history of contracts and vouch for their history of trustworthiness.
  4. It is possible for an upgradeable contract to become immutable and trustless.
  5. An upgradeable contract can have parts of it that are not upgradeable and so are partially immutable and trustless.

Immutability

In some cases immutable, trustless contracts are the right fit. This is the case when a contract is only needed for a short time or it is known ahead of time that there will never be any reason to change or improve it.

Middle Ground

Transparent contracts provide a middle ground between immutable trustless contracts that can’t be improved and upgradeable contracts that can’t be trusted.

Purposes

  1. Create upgradeable contracts that earn trust by showing a provable history of trustworthiness.
  2. Document the development of contracts so their development and change is provably public and can be understood.
  3. Create upgradeable contracts that can become immutable in the future if desired.
  4. Create contracts that are not limited by a max size.

Benefits & Use Cases

This standard is for use cases that benefit from the following:

  1. The ability to add, replace or remove multiple functions of a contract atomically (at the same time).
  2. Each time a function is added, replaced or removed, it is documented with events.
  3. Build trust over time by showing all changes made to a contract.
  4. Unlimited contract size.
  5. The ability to query information about functions currently supported by the contract.
  6. One contract address that provides all needed functionality and never needs to be replaced by another contract address.
  7. The ability for a contract to be upgradeable for a time, and then become immutable.
  8. Add trustless guarantees to a contract with “unchangeable functions”.

New Software Possibilities

This standard enables a form of contract version control software to be written.

Software and user interfaces can be written to filter the FunctionUpdate and CommitMessage events of a contract address. Such software can show the full history of changes of any contract that implements this standard.

User interfaces and software can also use this standard to assist or automate changes of contracts.

Specification

Note: The solidity delegatecall opcode enables a contract to execute a function from another contract, but it is executed as if the function was from the calling contract. Essentially delegatecall enables a contract to “borrow” another contract’s function. Functions executed with delegatecall affect the storage variables of the calling contract, not the contract where the functions are defined.

General Summary

A transparent contract delegates or forwards function calls to it to other contracts using delegatecode.

A transparent contract has an updateContract function that enables multiple functions to be added, replaced or removed.

An event is emitted for every function that is added, replaced or removed so that all changes to a contract can be tracked in a standard way.

A transparent contract is a contract that implements and complies with the design points below.

Terms

  1. In this standard a delegate contract is a contract that a transparent contract fallback function forwards function calls to using delegatecall.
  2. In this standard an unchangeable function is a function that is defined directly in a transparent contract and so cannot be replaced or removed.

Design Points

A contract is a transparent contract if it implements the following design points:

  1. A transparent contract is a contract that contains a fallback function, a constructor, and zero or more unchangeable functions that are defined directly within it.
  2. The constructor of a transparent contract associates the updateContract function with a contract that implements the ERC1538 interface. The updateContract function can be an “unchangeable function” that is defined directly in the transparent contract or it can be defined in a delegate contract. Other functions can also be associated with contracts in the constructor.
  3. After a transparent contract is deployed functions are added, replaced and removed by calling the updateContract function.
  4. The updateContract function associates functions with contracts that implement those functions, and emits the CommitMessage and FunctionUpdate events that document function changes.
  5. The FunctionUpdate event is emitted for each function that is added, replaced or removed. The CommitMessage event is emitted one time for each time the updateContract function is called and is emitted after any FunctionUpdate events are emitted.
  6. The updateContract function can take a list of multiple function signatures in its _functionSignatures parameter and so add/replace/remove multiple functions at the same time.
  7. When a function is called on a transparent contract it executes immediately if it is an “unchangeable function”. Otherwise the fallback function is executed. The fallback function finds the delegate contract associated with the function and executes the function using delegatecall. If there is no delegate contract for the function then execution reverts.
  8. The source code of a transparent contract and all delegate contracts used by it are publicly viewable and verified.

The transparent contract address is the address that users interact with. The transparent contract address never changes. Only delegate addresses can change by using the updateContracts function.

Typically some kind of authentication is needed for adding/replacing/removing functions from a transparent contract, however the scheme for authentication or ownership is not part of this standard.

Example

Here is an example of an implementation of a transparent contract. Please note that the example below is an example only. It is not the standard. A contract is a transparent contract when it implements and complies with the design points listed above.

pragma solidity ^0.5.7;

contract ExampleTransparentContract {
  // owner of the contract
  address internal contractOwner;
  event OwnershipTransferred(address indexed previousOwner, address indexed newOwner);

  // maps functions to the delegate contracts that execute the functions
  // funcId => delegate contract
  mapping(bytes4 => address) internal delegates;

  // maps each function signature to its position in the funcSignatures array.
  // signature => index+1
  mapping(bytes => uint256) internal funcSignatureToIndex;
    
  event CommitMessage(string message);
  event FunctionUpdate(bytes4 indexed functionId, address indexed oldDelegate, address indexed newDelegate, string functionSignature);
  
  // this is an example of an "unchangeable function".
  // return the delegate contract address for the supplied function signature
  function delegateAddress(string calldata _functionSignature) external view returns(address) {
    require(funcSignatureToIndex[bytes(_functionSignature)] != 0, "Function signature not found.");
    return delegates[bytes4(keccak256(bytes(_functionSignature)))];
  }
  
  // add a function using the updateContract function
  // this is an internal helper function
  function addFunction(address _erc1538Delegate, address contractAddress, string memory _functionSignatures, string memory _commitMessage) internal {    
    // 0x03A9BCCF == bytes4(keccak256("updateContract(address,string,string)"))
    bytes memory funcdata = abi.encodeWithSelector(0x03A9BCCF, contractAddress, _functionSignatures, _commitMessage);
    bool success;
    assembly {
      success := delegatecall(gas, _erc1538Delegate, add(funcdata, 0x20), mload(funcdata), funcdata, 0)
    }
    require(success, "Adding a function failed");   
  }

  constructor(address _erc1538Delegate) public {
    contractOwner = msg.sender;
    emit OwnershipTransferred(address(0), msg.sender);

    // adding ERC1538 updateContract function
    bytes memory signature = "updateContract(address,string,string)";
    bytes4 funcId = bytes4(keccak256(signature));
    delegates[funcId] = _erc1538Delegate;
    emit FunctionUpdate(funcId, address(0), _erc1538Delegate, string(signature));
    emit CommitMessage("Added ERC1538 updateContract function at contract creation");
	
    // associate "unchangeable functions" with this transparent contract address
    // prevents function selector clashes with delegate contract functions
    // uses the updateContract function
    string memory functions = "delegateAddress(string)";
    addFunction(_erc1538Delegate, address(this), functions, "Associating unchangeable functions");
	
    // adding ERC1538Query interface functions
    functions = "functionByIndex(uint256)functionExists(string)delegateAddresses()delegateFunctionSignatures(address)functionById(bytes4)functionBySignature(string)functionSignatures()totalFunctions()";    
    // "0x01234567891011121314" is an example address of an ERC1538Query delegate contract
    addFunction(_erc1538Delegate, 0x01234567891011121314, functions, "Adding ERC1538Query functions");
    
    // additional functions could be added at this point
  }

  // Making the fallback function payable makes it work for delegate contract functions 
  // that are payable and not payable.
  function() external payable {
    // Delegate every function call to a delegate contract
    address delegate = delegates[msg.sig];
    require(delegate != address(0), "Function does not exist.");
    assembly {
      let ptr := mload(0x40)
      calldatacopy(ptr, 0, calldatasize)
      let result := delegatecall(gas, delegate, ptr, calldatasize, 0, 0)
      let size := returndatasize
      returndatacopy(ptr, 0, size)
      switch result
      case 0 {revert(ptr, size)}
      default {return (ptr, size)}
    }
  }
}

As can be seen in the above example, every function call is delegated to a delegate contract, unless the function is defined directly in the transparent contract (making it an unchangeable function).

The constructor function adds the updateContract function to the transparent contract, which is then used to add other functions to the transparent contract.

Each time a function is added to a transparent contract the events CommitMessage and FunctionUpdate are emitted to document exactly what functions where added or replaced and why.

The delegate contract that implements the updateContract function implements the following interface:

ERC1538 Interface

pragma solidity ^0.5.7;

/// @title ERC1538 Transparent Contract Standard
/// @dev Required interface
///  Note: the ERC-165 identifier for this interface is 0x61455567
interface ERC1538 {
  /// @dev This emits when one or a set of functions are updated in a transparent contract.
  ///  The message string should give a short description of the change and why
  ///  the change was made.
  event CommitMessage(string message);
  
  /// @dev This emits for each function that is updated in a transparent contract.
  ///  functionId is the bytes4 of the keccak256 of the function signature.
  ///  oldDelegate is the delegate contract address of the old delegate contract if
  ///  the function is being replaced or removed.
  ///  oldDelegate is the zero value address(0) if a function is being added for the
  ///  first time.
  ///  newDelegate is the delegate contract address of the new delegate contract if 
  ///  the function is being added for the first time or if the function is being 
  ///  replaced.
  ///  newDelegate is the zero value address(0) if the function is being removed.
  event FunctionUpdate(
    bytes4 indexed functionId, 
    address indexed oldDelegate, 
    address indexed newDelegate, 
    string functionSignature
  );

  /// @notice Updates functions in a transparent contract.
  /// @dev If the value of _delegate is zero then the functions specified 
  ///  in _functionSignatures are removed.
  ///  If the value of _delegate is a delegate contract address then the functions 
  ///  specified in _functionSignatures will be delegated to that address.
  /// @param _delegate The address of a delegate contract to delegate to or zero
  ///        to remove functions.      
  /// @param _functionSignatures A list of function signatures listed one after the other
  /// @param _commitMessage A short description of the change and why it is made
  ///        This message is passed to the CommitMessage event.          
  function updateContract(address _delegate, string calldata _functionSignatures, string calldata _commitMessage) external;  
}

Function Signatures String Format

The text format for the _functionSignatures parameter is simply a string of function signatures. For example: "myFirstFunction()mySecondFunction(string)" This format is easy to parse and is concise.

Here is an example of calling the updateContract function that adds the ERC721 standard functions to a transparent contract:

functionSignatures = "approve(address,uint256)balanceOf(address)getApproved(uint256)isApprovedForAll(address,address)ownerOf(uint256)safeTransferFrom(address,address,uint256)safeTransferFrom(address,address,uint256,bytes)setApprovalForAll(address,bool)transferFrom(address,address,uint256)"
tx = await transparentContract.updateContract(erc721Delegate.address, functionSignatures, "Adding ERC721 functions");

Removing Functions

Functions are removed by passing address(0) as the first argument to the updateContract function. The list of functions that are passed in are removed.

Source Code Verification

The transparent contract source code and the source code for the delegate contracts should be verified in a provable way by a third party source such as etherscan.io.

Function Selector Clash

A function selector clash occurs when a function is added to a contract that hashes to the same four-byte hash as an existing function. This is unlikely to occur but should be prevented in the implementation of the updateContract function. See the reference implementation of ERC1538 to see an example of how function clashes can be prevented.

ERC1538Query

Optionally, the function signatures of a transparent contract can be stored in an array in the transparent contract and queried to get what functions the transparent contract supports and what their delegate contract addresses are.

The following is an optional interface for querying function information from a transparent contract:

pragma solidity ^0.5.7;

interface ERC1538Query {
    
  /// @notice Gets the total number of functions the transparent contract has.
  /// @return The number of functions the transparent contract has,
  ///  not including the fallback function.
  function totalFunctions() external view returns(uint256);
	
  /// @notice Gets information about a specific function
  /// @dev Throws if `_index` >= `totalFunctions()`
  /// @param _index The index position of a function signature that is stored in an array
  /// @return The function signature, the function selector and the delegate contract address
  function functionByIndex(uint256 _index) 
    external 
    view 
    returns(
      string memory functionSignature, 
      bytes4 functionId, 
      address delegate
    );
	
  /// @notice Checks to see if a function exists
  /// @param The function signature to check
  /// @return True if the function exists, false otherwise
  function functionExists(string calldata _functionSignature) external view returns(bool);
	
  /// @notice Gets all the function signatures of functions supported by the transparent contract
  /// @return A string containing a list of function signatures
  function functionSignatures() external view returns(string memory);
	
  /// @notice Gets all the function signatures supported by a specific delegate contract
  /// @param _delegate The delegate contract address
  /// @return A string containing a list of function signatures
  function delegateFunctionSignatures(address _delegate) external view returns(string memory);
	
  /// @notice Gets the delegate contract address that supports the given function signature
  /// @param The function signature
  /// @return The delegate contract address
  function delegateAddress(string calldata _functionSignature) external view returns(address);
	
  /// @notice Gets information about a function
  /// @dev Throws if no function is found
  /// @param _functionId The id of the function to get information about
  /// @return The function signature and the contract address
  function functionById(bytes4 _functionId) 
    external 
    view 
    returns(
      string memory signature, 
      address delegate
    );
	
  /// @notice Get all the delegate contract addresses used by the transparent contract
  /// @return An array of all delegate contract addresses
  function delegateAddresses() external view returns(address[] memory);
}

See the reference implementation of ERC1538 to see how this is implemented.

The text format for the list of function signatures returned from the delegateFunctionSignatures and functionSignatures functions is simply a string of function signatures. Here is an example of such a string: "approve(address,uint256)balanceOf(address)getApproved(uint256)isApprovedForAll(address,address)ownerOf(uint256)safeTransferFrom(address,address,uint256)safeTransferFrom(address,address,uint256,bytes)setApprovalForAll(address,bool)transferFrom(address,address,uint256)"

How To Deploy A Transparent Contract

  1. Create and deploy to a blockchain a contract that implements the ERC1538 interface. You can skip this step if there is already such a contract deployed to the blockchain.
  2. Create your transparent contract with a fallback function as given above. Your transparent contract also needs a constructor that adds the updateContract function.
  3. Deploy your transparent contract to a blockchain. Pass in the address of the ERC1538 delegate contract to your constructor if it requires it.

See the reference implementation for examples of these contracts.

Wrapper Contract for Delegate Contracts that Depend on Other Delegate Contracts

In some cases some delegate contracts may need to call external/public functions that reside in other delegate contracts. A convenient way to solve this problem is to create a contract that contains empty implementations of functions that are needed and import and extend this contract in delegate contracts that call functions from other delegate contracts. This enables delegate contracts to compile without having to provide implementations of the functions that are already given in other delegate contracts. This is a way to save gas, prevent reaching the max contract size limit, and prevent duplication of code. This strategy was given by @amiromayer. See his comment for more information. Another way to solve this problem is to use assembly to call functions provided by other delegate contracts.

Decentralized Authority

It is possible to extend this standard to add consensus functionality such as an approval function that multiple different people call to approve changes before they are submitted with the updateContract function. Changes only go into effect when the changes are fully approved. The CommitMessage and ` FunctionUpdate` events should only be emitted when changes go into effect.

Security

This standard refers to owner(s) as one or more individuals that have the power to add/replace/remove functions of an upgradeable contract.

General

The owners(s) of an upgradeable contract have the ability to alter, add or remove data from the contract’s data storage. Owner(s) of a contract can also execute any arbitrary code in the contract on behalf of any address. Owners(s) can do these things by adding a function to the contract that they call to execute arbitrary code. This is an issue for upgradeable contracts in general and is not specific to transparent contracts.

Note: The design and implementation of contract ownership is not part of this standard. The examples given in this standard and in the reference implementation are just examples of how it could be done.

Unchangeable Functions

“Unchangeable functions” are functions defined in a transparent contract itself and not in a delegate contract. The owner(s) of a transparent contract are not able to replace these functions. The use of unchangeable functions is limited because in some cases they can still be manipulated if they read or write data to the storage of the transparent contract. Data read from the transparent contract’s storage could have been altered by the owner(s) of the contract. Data written to the transparent contract’s storage can be undone or altered by the owner(s) of the contract.

In some cases unchangeble functions add trustless guarantees to a transparent contract.

Transparency

Contracts that implement this standard emit an event every time a function is added, replaced or removed. This enables people and software to monitor the changes to a contract. If any bad acting function is added to a contract then it can be seen. To comply with this standard all source code of a transparent contract and delegate contracts must be publicly available and verified.

Security and domain experts can review the history of change of any transparent contract to detect any history of foul play.

Rationale

String of Function Signatures Instead of bytes4[] Array of Function Selectors

The updateContract function takes a string list of functions signatures as an argument instead of a bytes4[] array of function selectors for three reasons:

  1. Passing in function signatures enables the implementation of updateContract to prevent selector clashes.
  2. A major part of this standard is to make upgradeable contracts more transparent by making it easier to see what has changed over time and why. When a function is added, replaced or removed its function signature is included in the FunctionUpdate event that is emitted. This makes it relatively easy to write software that filters the events of a contract to display to people what functions have been added/removed and changed over time without needing access to the source code or ABI of the contract. If only four-byte function selectors were provided this would not be possible.
  3. By looking at the source code of a transparent contract it is not possible to see all the functions that it supports. This is why the ERC1538Query interface exists, so that people and software have a way to look up and examine or show all functions currently supported by a transparent contract. Function signatures are used so that ERC1538Query functions can show them.

Gas Considerations

Delegating function calls does have some gas overhead. This is mitigated in two ways:

  1. Delegate contracts can be small, reducing gas costs. Because it costs more gas to call a function in a contract with many functions than a contract with few functions.
  2. Because transparent contracts do not have a max size limitation it is possible to add gas optimizing functions for use cases. For example someone could use a transparent contract to implement the ERC721 standard and implement batch transfer functions from the ERC1412 standard to help reduce gas (and make batch transfers more convenient).

Storage

The standard does not specify how data is stored or organized by a transparent contract. But here are some suggestions:

Inherited Storage

  1. The storage variables of a transparent contract consist of the storage variables defined in the transparent contract source code and the source code of delegate contracts that have been added.

  2. A delegate contract can use any storage variable that exists in a transparent contract as long as it defines within it all the storage variables that exist, in the order that they exist, up to and including the ones being used.

  3. A delegate contract can create new storage variables as long as it has defined, in the same order, all storage variables that exist in the transparent contract.

Here is a simple way inherited storage could be implemented:

  1. Create a storage contract that contains the storage variables that your transparent contract and delegate contracts will use.
  2. Make your delegate contracts inherit the storage contract.
  3. If you want to add a new delegate contract that adds new storage variables then create a new storage contract that adds the new storage variables and inherits from the old storage contract. Use your new storage contract with your new delegate contract.
  4. Repeat steps 2 or 3 for every new delegate contract.

Unstructured Storage

Assembly is used to store and read data at specific storage locations. An advantage to this approach is that previously used storage locations don’t have to be defined or mentioned in a delegate contract if they aren’t used by it.

Eternal Storage

Data can be stored using a generic API based on the type of data. See ERC930 for more information.

Becoming Immutable

It is possible to make a transparent contract become immutable. This is done by calling the updateContract function to remove the updateContract function. With this gone it is no longer possible to add, replace and remove functions.

Versions of Functions

Software or a user can verify what version of a function is called by getting the delegate contract address of the function. This can be done by calling the delegateAddress function from the ERC1538Query interface if it is implemented. This function takes a function signature as an argument and returns the delegate contract address where it is implemented.

Best Practices, Tools and More Information

More information, tools, tutorials and best practices concerning transparent contracts need to be developed and published.

Below is a growing list of articles concerning transparent contracts and their use. If you have an article about transparent contracts you would like to share then please submit a comment to this issue about it to get it added.

ERC1538: Future Proofing Smart Contracts and Tokens

The ERC1538 improving towards the “transparent contract” standard

Inspiration

This standard was inspired by ZeppelinOS’s implementation of Upgradeability with vtables.

This standard was also inspired by the design and implementation of the Mokens contract from the Mokens project. The Mokens contract has been upgraded to implement this standard.

Backwards Compatibility

This standard makes a contract compatible with future standards and functionality because new functions can be added and existing functions can be replaced or removed.

This standard future proofs a contract.

Implementation

A reference implementation of this standard is given in the transparent-contracts-erc1538 repository.

Copyright and related rights waived via CC0.