Deploy a Custom Protocol in Tezos | Core Engineering at METASTATE

Requirements
Prepping the environment
Constructing a dummy protocol
Deploying your protocol

The Tezos white paper takes the view that blockchains can be viewed as
the synthesis of three distinct protocols; namely network,
transaction, and consensus. However, for many blockchains the network
protocol is just a detail that must be figured out before being able
to deploy the more novel consensus and/or transaction systems. Tezos
realized this and created a generic shell that is agnostic to these
two protocols. Thus, it may be advantageous for such chains to first
model themselves inside of Tezos before dealing with such
plumbing. This, however, is predicated on the deployment and creation
of the protocols being easier than dealing with the networking plumbing.

This blogpost sets out to show that deploying a custom protocol in
Tezos is rather simple. A future post will be a detailed walkthrough
of creating a protocol.

Requirements

For active development, Tezos currently supports

macOS/x86_64
Linux/armv7h (32 bits) (Raspberry Pi3, etc.)
Linux/aarch64 (64 bits) (Raspberry Pi3, etc.)

Currently Windows development is not supported.

For deployment of Tezos, a working installation of OCaml, Dune, and
Opam are required.

Prepping the environment

We will first need to clone the Tezos code base.

git clone https://gitlab.com/tezos/tezos.git
cd tezos/

Now I would suggest running make build-dev-dep.

build-dev-deps installs Merlin which is useful for OCaml
development.
This will also create an Opam switch with the Tezos custom
compiler.

Run eval $(opam env), to properly go to the correct OCaml compiler
version.

Make sure to run this before running any other command, or
else there will be serious issues that will cause future commands
to not work.

We can now run make install, and finish the environment
preparations.

Constructing a dummy protocol

For the purposes of this tutorial, we will be copying the proto_demo
protocol, and modifying the bare minimum to make a distinct
protocol. The new protocol's name is proto_custom_demo, feel free to
replace any mention of proto_custom_demo with your own protocols
name. This section will also cover the essentials for getting the
development environment working properly.

Copy the demo protocol over to the new folder.
```
cp -r src/proto_demo src/proto_custom_demo
```

Cd into the new directory

cd src/proto_custom_demo/

The new directory should look like this

.
├── lib_client
│   ├── client_proto_main.ml
│   ├── client_proto_main.mli
│   ├── dune
│   ├── dune-project
│   ├── proto_demo.ml
│   ├── tezos-client-demo.install
│   └── tezos-client-demo.opam
└── lib_protocol
    ├── dune -> ../../lib_protocol_compiler/dune_protocol
    ├── dune.inc
    ├── dune-project
    ├── error.ml
    ├── main.ml
    ├── main.mli
    ├── services.ml
    ├── services.mli
    ├── tezos-embedded-protocol-demo.install
    ├── tezos-embedded-protocol-demo.opam
    ├── TEZOS_PROTOCOL
    ├── tezos-protocol-demo.install
    └── tezos-protocol-demo.opam

2 directories, 20 files

the .install files can be safely deleted!

In this directory we will need to change/remove a few file names to make this
protocol distinct, namely:

lib_protocol
- tezos-embedded-protocol-demo.install
- tezos-embedded-protocol-demo.opam
- tezos-protocol-demo.install
- tezos-protocol-demo.opam
lib_client
- tezos-client-demo.install
- tezos-client-demo.opam

Thus we need to run

  mv lib_protocol/tezos-protocol-demo.opam lib_protocol/tezos-protocol-custom-demo.opam
  mv lib_protocol/tezos-embedded-protocol-demo.opam lib_protocol/tezos-embedded-protocol-custom-demo.opam

  mv lib_client/tezos-client-demo.opam lib_client/tezos-client-custom-demo.opam

  rm lib_client/tezos-client-demo.install
  rm lib_protocol/tezos-embedded-protocol-demo.install

The Contents of some of these files need to be slightly altered as
well. In the dune file of lib_client, any mention of _demo
or -demo should be changed to _custom_demo and -custom-demo

sed -i 's/-demo/-custom-demo/g' lib_client/dune
sed -i 's/_demo/_custom_demo/g' lib_client/dune

The opam files in both directories will also need to be changed,
mentions of demo should be changed to custom_demo. lib_client
should also first change -demo to -custom-demo first to avoid a
naming error

sed -i 's/demo/custom_demo/' lib_protocol/tezos-embedded-protocol-custom-demo.opam
sed -i 's/demo/custom_demo/' lib_protocol/tezos-protocol-custom-demo.opam
sed -i 's/-demo/-custom-demo/' lib_client/tezos-client-custom-demo.opam
sed -i 's/ demo/ custom_demo/' lib_client/tezos-client-custom-demo.opam

The last thing that needs to be changed for now is the
lib_client/proto_demo.ml, namely

module Name = struct let name = "demo" end

module Proto = Tezos_protocol_demo.Functor.Make(Demo_environment)

module Name = struct let name = "custom_demo" end

module Proto = Tezos_protocol_custom_demo.Functor.Make(Demo_environment)

Now we need to generate a new dune.inc file, this will serve as the
injection of how the rest of Tezos sees your code, setups
environments, and allows auto completion to run. This file is
normally completely automatically generated, however since we
copied over proto-demo, we must first run a few sed commands
on it before it will actually change
```
sed -i 's/-demo/-custom-demo/' lib_protocol/dune.inc
sed -i 's/_demo/_custom_demo/' lib_protocol/dune.inc
```
We can now generate our new dune.inc file
```
dune build @lib_protocol//runtest_dune_template --auto-promote
```
Every time you make a new file, I'd suggest rerunning this last
command!
Now we should change lib_protocol/TEZOS_PROTOCOL, by default the file looks
something like this
```
{
    "hash": "ProtoDemoDemoDemoDemoDemoDemoDemoDemoDemoDemoD3c8k9",
    "modules": ["Error", "Services", "Main"]
}
```
For any new modules (i.e. new files), we will need to include them
here.

We will also need to change the hash of this JSON structure, as
proto_demo already has this hash registered. Note that we can not
insert an arbitrary hash, by say turning the 9 into 0, instead it
must be a valid hash.

Thankfully, Tezos comes with a script that can automatically
generate the hash for us.
```
tezos-protocol-compiler -hash-only ./lib_protocol/
```
This generates a hash of the protocol, replace the hash of json
file shown above with this newly generated hash.
Change lib_client/client_proto_main.ml to reflect the new hash.

The file contains the following line
```
let protocol =
  Protocol_hash.of_b58check_exn
    "ProtoDemoDemoDemoDemoDemoDemoDemoDemoDemoDemoD3c8k9"
```
Change it so it includes the new hash!

Finally, rerun

dune build @lib_protocol//runtest_dune_template --auto-promote

We will need to tell Tezos about our protocol now, we can do this
in multiple ways, first we'll cd back to the root directory and run
scripts/activate_protocol.sh
```
cd ../..
 ./scripts/activate_protocol.sh src/proto_custom_demo/
Link in the Node? (no if you want to test injection) (Y/n) Y
User-activated update in 3 blocks? (Y/n) n
```
- Like above you'll want to say yes to the first prompt and no
  to the second.
- This command however has some draw backs, in particular it'll add
  the following unwanted lines to src/bin_client/dune and
  src/bin_client/tezos-client.opam
  - "tezos-client-custom-demo-commands"
  - "tezos-baking-custom-demo-commands"
  - "tezos-client-custom-demo-commands"
  - tezos-baking-custom-demo-commands.registration
  - tezos-client-custom-demo-commands.registration
  For a fully functioning client, this may be ideal, however since
  our protocol is a dummy one, all we need is the
  tezos-client-custom-demo links.
  
  Go to these files and delete the offending lines.

Now the custom protocol should be able to be built!

Deploying your protocol

Now that we've constructed a protocol, all that is left is to test
it. First, let's go back to the root directory of the project and
rerun.

make build-dev-deps
make install

A good guide with more details on the following steps can be found
here.

create a new terminal and run.
```
./src/bin_node/tezos-sandboxed-node.sh 1 --connections 1
```
- This runs a sandboxed node for testing

On your previous terminal run.

eval `./src/bin_client/tezos-init-sandboxed-client.sh 1`

Now, if the instructions of the last section were followed
correctly, you should be able to see your protocol in the following
command

tezos-admin-client list protocols

ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK
ProtoDemoDemoDemoDemoDemoDemoDemoDemoDemoDemoD3c8k9
ProtoGenesisGenesisGenesisGenesisGenesisGenesk612im
Psi56CyzCAftZ7sFb8QoWYrme3a7ZHhstPoKRgvudwDFEVwrpH7

For me, the hash came out to be Psi56CyzCAftZ7sFb8QoWYrme3a7ZHhstPoKRgvudwDFEVwrpH7.

Inject the protocol.

# create some protocol parameters in a temporary file
# this needs to be proper arguments if your protocol is not a dummy one!
mkdir tmp && echo { } > tmp/protocol_parameters.json

# replace Psi56CyzCAftZ7sFb8QoWYrme3a7ZHhstPoKRgvudwDFEVwrpH7 with your protocol hash!
tezos-client activate protocol Psi56CyzCAftZ7sFb8QoWYrme3a7ZHhstPoKRgvudwDFEVwrpH7 \
             with fitness 5 and key activator and parameters tmp/protocol_parameters.json

Bake a block.
```
tezos-client bake
```
- for a real protocol, the command wouldn't be a simple bake, it
  would be what ever command you programmed bake to be.

Check the head to confirm you are on the new protocol.

tezos-client rpc get /chains/main/blocks/head/metadata


   The node you are connecting to claims to be running in a
                    Tezos TEST SANDBOX.
      Do NOT use your fundraiser keys on this network.
  You should not see this message if you are not a developer.

{ "protocol": "Psi56CyzCAftZ7sFb8QoWYrme3a7ZHhstPoKRgvudwDFEVwrpH7",
  "next_protocol": "Psi56CyzCAftZ7sFb8QoWYrme3a7ZHhstPoKRgvudwDFEVwrpH7",
  "test_chain_status": { "status": "not_running" }, "max_operations_ttl": 0,
  "max_operation_data_length": 0, "max_block_header_length": 42,
  "max_operation_list_length": [] }

You've now deployed your own protocol on Tezos.

Written by Jeremy Ornelas, core developer and researcher at Metastate.

Metastate has ceased its activities on the 31st of January 2021. The team members have joined a new company and working on a new project. If you're interested in distributed systems research and engineering and systems engineering in Rust, checkout the open positions at Heliax.

If you have any feedback or questions on this article, please do not hesitate to contact us: hello@heliax.dev.

Deploying a Custom Protocol in Tezos

Table of Contents

Requirements

Prepping the environment

Constructing a dummy protocol

Deploying your protocol