Integrating Simnode

Simnode supports both parachain and standalone runtimes. In order for simnode to be fully functional, Simnode needs to be integrated into both your runtime and node cli.

Runtime Integration

The only runtime integration necessary is to expose a runtime API. This enables the Simnode RPC to construct extrinsics that resemble transactions from an account, but contain an empty signature.

Why does simnode need this runtime API? Well two things, first off if you’re operating on a live chain state then the native code is most definitely going to be different from your runtime code. This means that the transaction version is potentially different, or the SignedExtra might have changed. By making the runtime produce the extrinsic itself, we insulate ourselves from these potential problems.

First in your runtime crate add the simnode runtime API dependency.

toml
simnode-runtime-api = { version = "1.6.0", default-features = false }

NOTE that simnode releases will always track polkadot releases.

Find the impl_runtime_apis! line in your runtime’s lib.rs file and add the following code:

rust
impl<RuntimeCall, AccountId>
	simnode_runtime_api::CreateTransactionApi<Block, RuntimeCall, AccountId> for Runtime
where
	RuntimeCall: codec::Codec,
	Block: sp_runtime::traits::Block,
	/// These trait bounds are just so that UncheckedExtrinsic can encode the AccountId
	AccountId: codec::Codec
		+ codec::EncodeLike<sp_runtime::AccountId32>
		+ Into<sp_runtime::AccountId32>
		+ Clone
		+ PartialEq
		+ scale_info::TypeInfo
		+ core::fmt::Debug,
{
	fn create_transaction(account: AccountId, call: RuntimeCall) -> Vec<u8> {
		// just some imports we'll need.
		use codec::Encode;
		use sp_core::sr25519;
		use sp_runtime::{generic::Era, traits::StaticLookup, MultiSignature};
		// let's fetch the account nonce from frame_system as we need to have a valid nonce for this extrinsic to be valid
		let nonce = frame_system::Pallet::<Runtime>::account_nonce(account.clone());
		// here we define the signed extras, this will be different depending on your runtime requirements
		// make this matches the [`SignedExtra`] type you've defined in your runtime
		let extra = (
			frame_system::CheckNonZeroSender::<Runtime>::new(),
			frame_system::CheckSpecVersion::<Runtime>::new(),
			frame_system::CheckTxVersion::<Runtime>::new(),
			frame_system::CheckGenesis::<Runtime>::new(),
			frame_system::CheckEra::<Runtime>::from(Era::Immortal),
			frame_system::CheckNonce::<Runtime>::from(nonce),
			frame_system::CheckWeight::<Runtime>::new(),
			pallet_transaction_payment::ChargeTransactionPayment::<Runtime>::from(0),
		);
		// most runtimes use the multi-signature type so they can support all kinds of signature schemes
		// notice that this signature is empty.
		let signature = MultiSignature::from(sr25519::Signature([0_u8; 64]));
		// converts the AccountId to the Address type the runtime expects.
		let address = AccountIdLookup::unlookup(account.into());
		// constructs the "signed" extrinsic
		let ext =
			generic::UncheckedExtrinsic::<Address, RuntimeCall, Signature, SignedExtra>::new_signed(
				call, address, signature, extra,
			);
		ext.encode()
	}
}

In cases where you’re trying to run simnode over live chain data, and the runtime onchain doesn’t yet expose this runtime API, simnode will still work. But it helps to be aware that you’ll have to configure simnode correctly in the next step

CLI Integration

This is a bit more involved, but the pay-offs are worth it. First we’ll need to bring in the simnode library into your node crate. Depending on what kind of consensus mechanism your substrate-based blockchain uses, you’ll need to enable the appropriate feature as well.

toml
# features must be one of parachain, babe or aura
sc-simnode = { version = "1.6.0", features = ["parachain"] }

Next we’ll need to implement the sc_simnode::ChainInfo for your runtime. Add this code to the bottom of your command.rs file.

rust
pub struct RuntimeInfo;

impl sc_simnode::ChainInfo for RuntimeInfo {
	// make sure you pass the opaque::Block here
	type Block = parachain_template_runtime::opaque::Block;
	// the runtime type
	type Runtime = parachain_template_runtime::Runtime;
	// the runtime api
	type RuntimeApi = parachain_template_runtime::RuntimeApi;
	// [`SignedExtra`] for your runtime
	type SignedExtras = parachain_template_runtime::SignedExtra;

	// Initialize the [`SignedExtra`] for your runtime, you'll notice I'm calling a pallet method here in
	// order to read from the runtime storage. This is possible becase this method is called in an externalities
	// provided environment. So feel free to read your runtime storage.
	fn signed_extras(
		from: <Self::Runtime as frame_system::pallet::Config>::AccountId,
	) -> Self::SignedExtras {
		let nonce = frame_system::Pallet::<Self::Runtime>::account_nonce(from);
		(
			frame_system::CheckNonZeroSender::<Self::Runtime>::new(),
			frame_system::CheckSpecVersion::<Self::Runtime>::new(),
			frame_system::CheckTxVersion::<Self::Runtime>::new(),
			frame_system::CheckGenesis::<Self::Runtime>::new(),
			frame_system::CheckEra::<Self::Runtime>::from(Era::Immortal),
			frame_system::CheckNonce::<Self::Runtime>::from(nonce),
			frame_system::CheckWeight::<Self::Runtime>::new(),
			pallet_transaction_payment::ChargeTransactionPayment::<Self::Runtime>::from(0),
		)
	}
}

The next steps will vary depending on if you have a parachain or standalone runtime. Toggle the appropriate drop down for your situation.

Parachain

You should have a function in your service.rs file called new_partial with the following signature:

rust
type ParachainExecutor = NativeElseWasmExecutor<ParachainNativeExecutor>;
type ParachainClient = TFullClient<Block, RuntimeApi, ParachainExecutor>;
type ParachainBackend = TFullBackend<Block>;
type ParachainBlockImport = TParachainBlockImport<Block, Arc<ParachainClient>, ParachainBackend>;

pub fn new_partial(
	config: &Configuration,
) -> Result<
	PartialComponents<
		ParachainClient,
		ParachainBackend,
		(),
		sc_consensus::DefaultImportQueue<Block, ParachainClient>,
		sc_transaction_pool::FullPool<Block, ParachainClient>,
		(ParachainBlockImport, Option<Telemetry>, Option<TelemetryWorkerHandle>),
	>,
	sc_service::Error,
> {}

We need to modify this function to be compatible with simnode. Currently, the function creates its own executor and passes it on to sc_service::new_full_parts. However, we need to change this so that the function takes the executor as a second parameter. This will require us to change the return type signatures to be generic over the executor parameter.

If done correctly, your function should now look like this:

rust
type ParachainClient<E> = TFullClient<Block, RuntimeApi, E>;

pub fn new_partial<E>(
	config: &Configuration,
	executor: E,
) -> Result<
	PartialComponents<
		ParachainClient<E>,
		ParachainBackend,
		sc_simnode::ParachainSelectChain<ParachainClient<E>>,
		sc_consensus::DefaultImportQueue<Block, ParachainClient<E>>,
		sc_transaction_pool::FullPool<Block, ParachainClient<E>>,
		(ParachainBlockImport<E>, Option<Telemetry>, Option<TelemetryWorkerHandle>),
	>,
	sc_service::Error,
>
where
	E: CodeExecutor + RuntimeVersionOf + 'static,
{}

You’ll also noticed we replaced the unit struct () with ParachainSelectChain<_> this is because sc-consensus-manual-seal requires a SelectChain implementation. Since parachains use the relay chain as it’s fork choice rule, we’ve provided a stub implementation that does nothing.

You can initialise it like so:

rust
use sc_simnode::parachain::ParachainSelectChain;

pub fn new_partial<E>(
	config: &Configuration,
	executor: E,
) -> Result<
	PartialComponents<
		ParachainClient<E>,
		ParachainBackend,
		sc_simnode::ParachainSelectChain<ParachainClient<E>>,
		sc_consensus::DefaultImportQueue<Block, ParachainClient<E>>,
		sc_transaction_pool::FullPool<Block, ParachainClient<E>>,
		(ParachainBlockImport<E>, Option<Telemetry>, Option<TelemetryWorkerHandle>),
	>,
	sc_service::Error,
>
where
	E: CodeExecutor + RuntimeVersionOf + 'static,
{
	// ... lines omitted
  
  let select_chain = ParachainSelectChain::new(client.clone());

	Ok(PartialComponents {
		backend,
		client,
		import_queue,
		keystore_container,
		task_manager,
		transaction_pool,
		select_chain,
		other: (block_import, telemetry, telemetry_worker_handle),
	})
}

You will need to fix the compiler errors that resulted from changing the signature of the new_partial function. Once that’s done, add this variant to your SubCommand enum in your cli.rs.

rust
/// Sub-commands supported by the collator.
#[derive(Debug, clap::Subcommand)]
pub enum Subcommand {
    // ... lines omitted
    
    /// Runs the node with signature verification overrides and manual seal.
    Simnode(sc_simnode::cli::SimnodeCli),
    
    // ... lines omitted
}

You’ll need to modify the run function in your command.rs , add this variant to the match statement in the function.

rust
pub fn run() -> Result<()> {
	match &cli.subcommand {
		// ... lines omitted

		Some(Subcommand::Simnode(cmd)) => {
			let runner = cli.create_runner(&cmd.run.normalize())?;
			let config = runner.config();
			// here we use simnode's custom executor
			let executor = sc_simnode::new_wasm_executor(&config);
			// pass the custom executor along
			let components = new_partial(config, executor)?;

			runner.run_node_until_exit(move |config| async move {
				let client = components.client.clone();
				let pool = components.transaction_pool.clone();
				// start simnode's subsystems
				let task_manager =
					sc_simnode::parachain::start_simnode::<RuntimeInfo, _, _, _, _, _>(
						sc_simnode::SimnodeParams {
							components,
							config,
							// you'll want this to be set to true if you want simnode
							// to create blocks for every transaction that enters the tx pool.
							// For cases where you want to manually send 
							// RPC requests before blocks are created, set this to false.
							instant: true,
							rpc_builder: Box::new(move |deny_unsafe, _| {
								// initialize your own custom RPCs here
								let client = client.clone();
								let pool = pool.clone();
								let full_deps = rpc::FullDeps { client, pool, deny_unsafe };
								let io =
									rpc::create_full(full_deps).expect("Rpc to be initialized");

								Ok(io)
							}),
						},
					)
					.await?;
				Ok(task_manager)
			})
		},

		// ... lines omitted
	}
}

AURA

You should have a function in your service.rs file called new_partial with the following signature:

rust
pub(crate) type FullClient =
	sc_service::TFullClient<Block, RuntimeApi, NativeElseWasmExecutor<ExecutorDispatch>>;
type FullBackend = sc_service::TFullBackend<Block>;
type FullSelectChain = sc_consensus::LongestChain<FullBackend, Block>;

pub fn new_partial(
	config: &Configuration,
) -> Result<
	sc_service::PartialComponents<
		FullClient,
		FullBackend,
		FullSelectChain,
		sc_consensus::DefaultImportQueue<Block, FullClient>,
		sc_transaction_pool::FullPool<Block, FullClient>,
		(
			sc_consensus_grandpa::GrandpaBlockImport<
				FullBackend,
				Block,
				FullClient,
				FullSelectChain,
			>,
			sc_consensus_grandpa::LinkHalf<Block, FullClient, FullSelectChain>,
			Option<Telemetry>,
		),
	>,
	ServiceError,
> {

We need to modify this function to be compatible with what simnode expects. Currently, the function creates its own executor and passes it on to sc_service::new_full_parts. However, we need to change this so that the function takes the executor as a second parameter. This will require us to change the return type signatures to be generic over the executor parameter.

If done correctly, your function should now look like this:

rust

type FullClient<E> = sc_service::TFullClient<Block, RuntimeApi, E>

pub fn new_partial<E>(
	config: &Configuration,
	executor: E,
) -> Result<
	sc_service::PartialComponents<
		FullClient<E>,
		FullBackend,
		FullSelectChain,
		sc_consensus::DefaultImportQueue<Block, FullClient<E>>,
		sc_transaction_pool::FullPool<Block, FullClient<E>>,
		(
			sc_consensus_grandpa::GrandpaBlockImport<
				FullBackend,
				Block,
				FullClient<E>,
				FullSelectChain,
			>,
			Option<Telemetry>,
			sc_consensus_grandpa::LinkHalf<Block, FullClient<E>, FullSelectChain>,
		),
	>,
	ServiceError,
>
where
	E: CodeExecutor + RuntimeVersionOf + 'static,
{}

You will need to fix the compiler errors that resulted from changing the signature of the new_partial function. Once that’s done, add this variant to your SubCommand enum in your cli.rs.

rust
/// Sub-commands supported by the collator.
#[derive(Debug, clap::Subcommand)]
pub enum Subcommand {
    // ... lines omitted
    
    /// Runs the node with signature verification overrides and manual seal.
    Simnode(sc_simnode::cli::SimnodeCli),
    
    // ... lines omitted
}

You’ll need to modify the run function in your command.rs , add this variant to the match statement in the function.

rust
pub fn run() -> Result<()> {
	match &cli.subcommand {
		// ... lines omitted

		Some(Subcommand::Simnode(cmd)) => {
			let runner = cli.create_runner(&cmd.run.normalize())?;
			let config = runner.config();
			// here we use simnode's custom executor
			let executor = sc_simnode::new_wasm_executor(&config);
			// pass the custom executor along
			let components = new_partial(config, executor)?;

			runner.run_node_until_exit(move |config| async move {
				let client = components.client.clone();
				let pool = components.transaction_pool.clone();
				// start simnode's subsystems
				let task_manager =
					sc_simnode::aura::start_simnode::<RuntimeInfo, _, _, _, _, _>(
						sc_simnode::SimnodeParams {
							components,
							config,
							// you'll want this to be set to true so simnode creates
							// blocks for every transaction that enters the tx pool.
							// For cases where you want to manually send 
							// RPC requests before blocks are created, set this to false.
							instant: true,
							rpc_builder: Box::new(move |deny_unsafe, _| {
								// initialize your own custom RPCs here
								let client = client.clone();
								let pool = pool.clone();
								let full_deps = rpc::FullDeps { client, pool, deny_unsafe };
								let io =
									rpc::create_full(full_deps).expect("Rpc to be initialized");

								Ok(io)
							}),
						},
					)
					.await?;
				Ok(task_manager)
			})
		},

		// ... lines omitted
	}
}

BABE

You should have a function in your service.rs file called new_partial with the following signature:

rust
/// The full client type definition.
pub type FullClient =
	sc_service::TFullClient<Block, RuntimeApi, NativeElseWasmExecutor<ExecutorDispatch>>;
type FullBackend = sc_service::TFullBackend<Block>;
type FullSelectChain = sc_consensus::LongestChain<FullBackend, Block>;
type FullGrandpaBlockImport =
	grandpa::GrandpaBlockImport<FullBackend, Block, FullClient, FullSelectChain>;

/// Creates a new partial node.
pub fn new_partial(
	config: &Configuration,
) -> Result<
	sc_service::PartialComponents<
		FullClient,
		FullBackend,
		FullSelectChain,
		sc_consensus::DefaultImportQueue<Block, FullClient>,
		sc_transaction_pool::FullPool<Block, FullClient>,
		(
			impl Fn(
				node_rpc::DenyUnsafe,
				sc_rpc::SubscriptionTaskExecutor,
			) -> Result<jsonrpsee::RpcModule<()>, sc_service::Error>,
			(
				sc_consensus_babe::BabeBlockImport<Block, FullClient, FullGrandpaBlockImport<E>>,
				grandpa::LinkHalf<Block, FullClient, FullSelectChain>,
				sc_consensus_babe::BabeLink<Block>,
			),
			grandpa::SharedVoterState,
			Option<Telemetry>,
		),
	>,
	ServiceError,
>
{}

If done correctly, your function should now look like this:

rust
/// Creates a new partial node.
pub fn new_partial<E>(
	config: &Configuration,
	executor: E,
) -> Result<
	sc_service::PartialComponents<
		FullClient<E>,
		FullBackend,
		FullSelectChain,
		sc_consensus::DefaultImportQueue<Block, FullClient<E>>,
		sc_transaction_pool::FullPool<Block, FullClient<E>>,
		(
			impl Fn(
				node_rpc::DenyUnsafe,
				sc_rpc::SubscriptionTaskExecutor,
			) -> Result<jsonrpsee::RpcModule<()>, sc_service::Error>,
			(
				sc_consensus_babe::BabeBlockImport<Block, FullClient<E>, FullGrandpaBlockImport<E>>,
				grandpa::LinkHalf<Block, FullClient<E>, FullSelectChain>,
				sc_consensus_babe::BabeLink<Block>,
			),
			grandpa::SharedVoterState,
			Option<Telemetry>,
		),
	>,
	ServiceError,
>
where
	E: CodeExecutor + RuntimeVersionOf + 'static,
{

You will need to fix the compiler errors that resulted from changing the signature of the new_partial function. Once that’s done, add this variant to your SubCommand enum in your cli.rs.

rust
/// Sub-commands supported by the collator.
#[derive(Debug, clap::Subcommand)]
pub enum Subcommand {
    // ... lines omitted
    
    /// Runs the node with signature verification overrides and manual seal.
    Simnode(sc_simnode::cli::SimnodeCli),
    
    // ... lines omitted
}

You’ll need to modify the run function in your command.rs , add this variant to the match statement in the function.

rust
pub fn run() -> Result<()> {
	match &cli.subcommand {
		// ... lines omitted

		Some(Subcommand::Simnode(cmd)) => {
			let runner = cli.create_runner(&cmd.run.normalize())?;
			let config = runner.config();
			// here we use simnode's custom executor
			let executor = sc_simnode::new_wasm_executor(&config);
		// re-assemble partial components to what simnode expects
		let PartialComponents {
				client,
				backend,
				task_manager,
				import_queue,
				keystore_container,
				select_chain,
				transaction_pool,
				other: (rpc_builder, (block_import, _, babe_link), _, telemetry),
			} = new_partial(&config, executor)?;

			let components = PartialComponents {
				client,
				backend,
				task_manager,
				import_queue,
				keystore_container,
				select_chain,
				transaction_pool,
				other: (block_import, telemetry, babe_link),
			};

			runner.run_node_until_exit(move |config| async move {
				let client = components.client.clone();
				let pool = components.transaction_pool.clone();
				// start simnode's subsystems
				let task_manager =
					sc_simnode::babe::start_simnode::<RuntimeInfo, _, _, _, _, _>(
						sc_simnode::SimnodeParams {
							components,
							config,
							// you'll want this to be set to true so simnode creates
							// blocks for every transaction that enters the tx pool.
							// For special cases where you want to manually send 
							// RPC requests before blocks are created, set this to false.
							instant: true,
							rpc_builder: Box::new(rpc_builder),
						},
					)
					.await?;
				Ok(task_manager)
			})
		},

		// ... lines omitted
	}
}

Now that you’ve integrated simnode, compile your node and run the following command to witness simnode in it’s full glory.

bash
cargo build --release -p your-node
./target/release/your-node simnode --dev --state-pruning=archive --blocks-pruning=archive

You’ll notice that no blocks are being created and the node basically sits idle. This is exactly what should happen.

bash
2023-05-18 12:11:26 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:11:31 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:11:36 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:11:41 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:11:46 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:11:51 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:11:56 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:01 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:06 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:11 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:16 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:21 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:26 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:31 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0
2023-05-18 12:12:36 💤 Idle (0 peers), best: #0 (0xf2ac…7e42), finalized #0 (0xf2ac…7e42), ⬇ 0 ⬆ 0

Test that you can create blocks like so:

bash
curl http://127.0.0.1:9933 -d '{"id":29,"jsonrpc":"2.0","method":"engine_createBlock","params":[true, true]}' -H "Content-Type: application/json"
{"jsonrpc":"2.0","result":{"hash":"0xbf1120ffcc74ab53811f1c2ca27edcf61dc30546a44b5141c968b43581432278","aux":{"header_only":false,"clear_justification_requests":false,"needs_justification":false,"bad_justification":false,"is_new_best":true}},"id":29}

You should see your node output something along the following lines:

bash
2023-05-18 12:08:50 Accepting new connection 1/100
2023-05-18 12:08:50 🙌 Starting consensus session on top of parent 0xf2ac8c19f4f0138303250778ec575d1ad9cb7dfc40f41e359acd28dc4aee7e42
2023-05-18 12:08:50 🎁 Prepared block for proposing at 1 (0 ms) [hash: 0xf7dce7803a59d35f3e99f2c2a2d5e841a681349e4c863cd86dae452a22259494; parent_hash: 0xf2ac…7e42; extrinsics (2): [0x71a8…f4a4, 0x61d2…b836]]
2023-05-18 12:08:50 ✨ Imported #1 (0xf7dc…9494)

And that's it! You've successfully integrated Simnode into your substrate node! 🥳🎉🎊.

Next we’ll look at how to leverage simnode for writing integration tests for your runtime.