Shinami

July 6, 2023

Question

• feature request, display Move source code in Sui Explorer.
• We are seeing our users (application developers on Sui) run into issues with scaling their use of AdminCap (e.g. for minting) and in that process, also getting their objects locked. We were planning to work on solutions in this area but wanted to see: are there plans in the works on the Mysten side as well? How should we collaborate if so?

Discussion

• Explorer shows the compiled code of the Package, but it is not readable as compiled code. Sometimes publishers may include GitHub link to source code. Missing link to source code for someone to verify the source code vs just have to trust. Can it be an option for those publishing package so that the package is verifiable from the Explorer.
• Worked on this but then took a step back. Aptos’ manifest also goes on chain as an example. In a few weeks there should be a proof of concept of source code for the system packages. Through off-chain indexing, if indexer is given source code, the indexer can run verification to match, and provide the link. Gives a way to prototype of this API so that the Explorer knows how to access source code. Could also consider opening this up. In the sui cli there is a verify source command and will leverage the mechanism, and it recompiles the source code. The Move compiler also conducts dependency verification. The API the Explorer sees is exposed by Indexer - here is the Package and give me the source link for it.
• 2 availability questions
  • Make API available to anyone to use
  • Make it possible for anyone to stand up a service to provide a verified source
• Concurrent use of a single object due to how Sui is designed. A lot of folks did not realize they have to explicitly manage concurrent execution for an owned object. This is manifested most severely in AdminCap. It is a well-adopted pattern, as a publisher when you publish a package you get a global singleton to signify the authority. A lot of operations require AdminCap and it is easy to run into locking. While can help educate builders and to implement server-side locking, is there a more generalized solution. As a RPC provider, can implement object-based locking for the customers. Can sequence transactions that touch the same objects. Conceptually it is very easy to work around this problem. Can also create a second delegate and control multiple lanes.
• Did previously consider native equivocation protection on the full node, but opted to push this onto the client side and have logic in the SDK to do the same thing. Can have as many capabilities as we have workers. Is it possible to have a framework level library to codify this pattern? The two patterns are a duplicate pattern and a delegate pattern. There are good reasons to use one vs another. Framework can get shoehorned. The long-term direction is to make full nodes more stateless over time. Many users or clients are actually ‘servers’ and as they scale out they need to make sure that there is no overlap since locking is handled by the client.
• Related future things: send to object. Can construct an object as an inbox and it can be used as a message queue for multiple servers to synchronize. Multiple servers with different keys can synchronize. Another one is fast unlock. Can go to consensus just to unblock that one object vs having to wait a day. This is highly desired given how frequent people run into these issue.
• Custom indexing/custom RPCs. Kiosk is the most requested, what is in a Kiosk. Unwrapped objects is another thing people. have asked for. Current plan is to stabilize indexer for others to add their own indexing passes. First step is need to index the Kiosk data. Some custom RPCs can be very custom for the dapp. A Kiosk would be more universal across games and marketplaces.
• For full node to expose a resummable and reliable data hose for the chain. If the consumer has not finished, they can reconnect and reconsume. Full node needs to be aware that certain things has not been consumed, and need to hold on to the data before it gets pruned. This fire hose can enable people to index it in however way they can. Today’s workaround is getCheckpoint polling. But suffers from the fact that full node is not aware of the consumption and can prune the data. So today need to disable pruning with getCheckpoint. Ideally this can be a push fashion, and Kafka is a good infra for this. If there is a full node or Kafka integration this can be very powerful. Full node just makes sure that every checkpoint and txn block has been pushed, and then full node can move on. Most solutions can be developed behind this in a language-agnostic way. The full node and Kafka connector can be done in Rust. The downstream system can have many different requirements, may be single node or distributed, may have many different language needs. This is the ideal way this can work and benefit the community.
• Prefer for there to be a standard RPC, at the interface level this is how you query Kiosk, and can follow the standard. If there is a major part that is decoupled from the underlying implementation can be open sourced. For RPC hoping to provide an interface that is extensible without standardization, but this doesn’t negate standardization.
• Archival tends to imply a single machine. Goal is to keep data in a semi-efficiently-queryable storage format. Have relational data in a Postgres, have historical data in a datalake-ish solution, and stateless server nodes that knows what data sources to hit depending what data is being queried.
• Request for RPC that returns specific characteristics of the URI similar to protocol config, but can return rate limits. So it can be used to do client side throttling before rate limit requests. This would be difficult to standardize, looking for weighted compute units called CUPS. This could change between versions. Also do custom rate limits based on the API key. Such a RPC works for public end points if they want to run across custom end points. This type of introspection RPC makes sense but is difficult to standardize. Can an introspection API benefit bad actors. Can txn payload sizes be standardized? If full node can be configured then the introspection can expose the full node configuration.

Noted Areas of Improvement

• Better documentation for handling concurrent owned object access and in particular AdminCap