Manage Whitelists Like a Pro
All source code for this example can be found here, fork away!
In the web3 industry, employing a whitelist to control access to specific parts of your protocol is a widely adopted practice. Implementing this involves restricting access to certain functions within our smart contract, limiting it solely to a select few wallet addresses.
For gas saving reasons, the most efficient way of doing this is by using a merkle tree.
The reason we use a merkle tree is because storing data on-chain is expensive. Meaning, for every byte we store or change, the more gas we pay for that state-change transaction. If we didn't use a merkle tree, but instead we just set a huge mapping of, say, a thousand whitelisted addresses we would pay a ton of gas. And the longer the list, the more gas we pay.
It would be a lot cheaper to set one hash, only once, no matter the size of this list, and use that hash to prove that a user is in the list.
In order to do this we need to create a merkle tree using our whitelisted addresses as the leaf nodes. We then generate a root hash and store that in our contract. If the whitelist ever changes, we need to update this root.
Then, if a user wants to call a function in our contract that is for whitelisted users only, they need to send along a merkle proof. Think of this as a set of directions to follow from the root of the tree all the way to the correct leaf node for that user. If the address in that leaf node matches the address that sent the transaction ( msg.sender ), then we proceed with the transaction! Each proof is unique to each user.
So using our merkle tree is fairly simple, right? To summarize, we need to do 2 things:
Store the merkle root in our contract Generate a merkle proof using the user's address and send that along with other functions arguments. The contract can then use the root to verify the merkle proof, providing on-chain whitelisting. 🚀 Simple right?
There are simply too many points of failure in this process:
- Running a script manually to generate a merkle root is clunky and time consuming.
- Someone can add entries to the whitelist in an incorrect format, causing the script to fail unexpectedly
- You are not guaranteed that the person who owns the owner wallet will actually update the root, or do it in a timely manner.
- It can be difficult to know if the current root saved in the contract reflects the current state of the whitelist. (you can update the JSON file, but forget to generate the new root and call setRoot() on your contract! 🤦♂️)
- This means it can often take several team members to: update the whitelist, store it on S3 (or IPFS) and set the new root.
There is hope!
With some clever engineering and UX, we can make this process much easier to use and less prone to failure. We can:
- Move all the merkle tree logic from a script to an API route.
- Store the whitelist on S3 (or other storage provider)
- Create an easy-to-use admin panel for the owner wallet to use set the merkle root on-chain.
In this example we'll be using Next.js with typecsript, merkletreejs, ethers.js and zod as our validation library. All the code for this example can be found here. Now let's jump in to some code!
first start our project (nextjs 13 with app dir.)
And create this folder structure:
And before we start building our UI, let's get working on our 2 API routes:
First, let's get our logic together for fetching the whitelist file generating the merkle root.
This is what our whitelist file looks like:
Given the shape of our whitelist file, let's get some logic together for fetching and parsing this file in a type-safe way.
Here we're using zod to parse our json file and generate some types around it. We create a schema used to validate the file and export a type generated from that schema.
Here, the return type of fetchAndParseWhitelistFile is:
Now that we can safely fetch and parse our file, let's add some logic to generate the merkle root from an array of adresses:
And finally an api route we can call to fetch the root!
Let's test it by running our app locally
And we also need to generate a merkle proof given a user's address:
And an API route to call:
Again let's test it with bob's address (passed as a query parameter “address”):
Okay so now let's build some UI for our admin panel so that the owner wallet can manage the merkle root.
But first let's make some hooks to get the merkle root (from the whitelist)
Great! Now we know what the merkle root should be. But what is the latest merkle root stored on-chain? Let's create a hook to read that value from the contract.
Awesome! Now to the same file we can add a function to set the merkle root on-chain to what it should be according to the whitelist.
And finally let's create a simple UI the admin wallet can use to:
Check if the on-chain merkle root is up to date and if not, Set it to what it should be!
Out of Sync
Oh no I messed up the json file oops.
Now they are in sync
I know what you're thinking:
This is completely over-engineered! And you might be right… BUT when your protocol start growing at a rapid rate and you have several contracts each with their own whitelists, things can get hairy VERY quickly.
This solution is a lot of boilerplate, but it scales really well and cuts down on room for error which is exactly what you want when you have a discord full of mad people aping into your NFT project.
Hope this helps. Happy coding frens.