Tezos protocol upgrade development (part 1)

One of the key defining features of Tezos is to include the protocol upgrades in the protocol itself to create what has been dubbed as self-amending crypto-ledger. The upgrades are submitted through the blockchain, but before they get deployed, they have to pass multiple stages of votes, that also take place on the blockchain. In this post, we'll look at what encompasses the process of building a protocol upgrade.

Before you begin...

Say you have an idea for protocol change and you have the capabilities to implement it. Great! But there's of course more to a successful protocol upgrade than just writing the code that shouldn't be overlooked. Consider other teams (currently Nomadic Labs and us, Cryptium Labs) working on evolving the protocol and get in touch with them or open an issue on the main Tezos repository. This generates discussion around the changes you'd like to propose and you'll get some feedback. After all, your proposal will have to convince the stakeholders to vote for it, so it's good to put your design to the test early on. Also, it may be possible to bundle your changes together with other proposals, instead of competing with them (only one upgrade can pass the proposal stage at the time) and possibly get the upgrade deployed sooner.

Development prerequisites

The obvious part should be to learn OCaml. You'll want to be comfortable with both reading and writing OCaml. There are numerous resources available to help along the way:

• Real-world OCaml book
• Developing Applications with Objective Caml
• Functional Programming in OCaml
• Exercism OCaml track
• OCaml Discourse

The Tezos documentation covers many topics you'll want to be familiar with, especially in the developer section. Both the data encoding and the error monad libraries are used extensively. For protocol upgrade itself it's essential to read the protocol Alpha section and how the protocol is sandboxed (in the protocol, you can only use modules and interfaces that are exposed from lib_protocol_environment and while this might seem to be limiting, there are good reasons for it). You may gain better insight by also looking at the code as you read the documentation.

Make sure you can build from sources. For development, instead of make build-deps invoke make build-dev-deps to install Merlin for code completion and more.

If you're not already familiar with the Tezos client, you can explore the many available commands and RPC interface from within the sandbox mode (a local private network with configurable number of nodes), which is also very useful for quickly experimenting with the changes you've applied to the protocol.

Protocol code overview

For protocol upgrade, you can typically start your branch from master. Your proposal code starts off the protocol Alpha in src/proto_alpha/lib_protocol, which will contain the code of the latest active protocol (and indeed will have the code of your proposal after it's successfully voted and injected). This is the green eye of the octopus, described in more detail in the aforementioned protocol Alpha section.

In brief,

• Note that the TEZOS_PROTOCOL file defines the linking order of the contained modules and its grouping aids in understanding how the code is structured
• The Alpha_context module re-defines abstract modules typically by glueing together concrete *_repr (representation) and *_storage modules. For example, module Vote is comprised of Vote_repr and Vote_storage. Vote_repr contains data definitions and their encoding while Vote_storage exposes accessors to key-value stores
• All the storage accessors are defined in the Storage module using the many variants of storage signatures from Storage_sigs together with storage builders from Storage_functors
• Much of the protocol logic is built using the Alpha_context abstractions in Apply (protocol's main entry points), Baking and protocol Amendment modules
• Script_* modules deal with Michelson smart contracts, whose code is translated to an intermediate representation to be interpreted when a contract code is being run
• Finally, the *_services modules define the protocol RPC API

The Tezos documentation goes into more detail and we recommend that you get comfortable with it. We hope that this will give you a good introductory overview of the codebase and how to navigate it. If you have questions or get stuck on something, don't hesitate to reach out for help!

In the next part, we'll look at how to handle protocol upgrades that introduce structural changes, that require a data migration to bridge the changes from a previous protocol to a new one.

Written by Tomáš Zemanovič, core developer and researcher at Metastate.

Metastate has ceased its activities on the 31st of January 2021. The team members have joined a new company and working on a new project. If you're interested in distributed systems research and engineering and systems engineering in Rust, checkout the open positions at Heliax.

If you have any feedback or questions on this article, please do not hesitate to contact us: hello@heliax.dev.