Uploads and ANS-104 data items

When an item has Html, Image or File values, @seedprotocol/publish uploads their bytes to Arweave before it makes the attestations, and the attestations record the resulting Arweave IDs. An Ethereum key, normally the user's own wallet, signs one ANS-104 data item per file, and the upload API at uploadApiBaseUrl passes them to a bundler, which posts them to Arweave in bundles and pays for them. With payment: { storage: 'user' } the data items go straight to Turbo instead, where the user pays, and uploadApiBaseUrl can be left out (below).

Uploads go to Arweave mainnet, so the data is public and permanent and someone pays for it in real AR; see What "permanent" means today and Costs and who pays.

Seed's gateway

The Seed Protocol team runs its own private AR.io gateway, which makes uploads through it immediate and sponsored. Open-sourcing its code is on the roadmap. Apps outside Seed set uploadApiBaseUrl to an upload API they have access to, or have users pay for storage through Turbo with payment: { storage: 'user' } (below).

What publish sends

/batch goes to uploadApiBaseUrl. The data and status requests go to arweaveUploadVerificationBaseUrl, which defaults to uploadApiBaseUrl.

When the app sets authorizePublish and it returns a token, /batch requests carry it in an X-Seed-Publish-Authorization header, so an upload API can meter or bill each publish. See Approving publishes. A browser calling an upload API on another origin sends a CORS preflight first, and the upload API must list X-Seed-Publish-Authorization in its allowed headers or the browser blocks the request.

The bundler path

A publish signs with the publish wallet's data item signer unless you pass one (below). The only other setting is uploadApiBaseUrl.

ANS-104 data items

ANS-104 defines a data item: a small signed envelope of data and tags that isn't an Arweave transaction itself. A bundler collects many data items and posts them as the data of one L1 transaction, a bundle. Gateways that unpack the bundle then serve each item under its own ID.

A data item has:

The bytes and tags are built by the SDK's getPublishUploadData.

The ID is the SHA-256 hash of the signature, and the signature covers everything else.

Who signs

Pass either one to createPublish, or set it in the config as a fallback. With neither, the publish fails at creatingArweaveDataItems with "@seedprotocol/publish: uploading data items needs signDataItems or a dataItemSigner (pass one at createPublish or in the PublishProvider config). It must be an EOA's key: …". No Arweave key is involved.

The signer must produce plain ECDSA signatures that recover to its own public key, because that public key becomes the data item's owner. That rules out smart accounts, whose signatures their contract checks (ERC-1271, or ERC-6492 before deployment).

ConnectButton registers an EOA as the publish wallet's data item signer: the in-app wallet, which is the ManagedAccount's admin. Every publish as that wallet's publisherAddress uses it unless you pass one or set one in the config. dataItemSigner has the order a publish picks a signer in.

The data item's owner is always a key the user or app controls, not Seed. It isn't the author either; see who signs what.

/batch

The publish package sends all the data items in one POST ${uploadApiBaseUrl}/api/upload/arweave/batch with Content-Type: application/octet-stream, and a body of:

item count       uint32, big-endian
for each item:
  length         uint32, big-endian
  bytes          the signed data item

Publish reads total, succeeded and failed from the JSON response. It treats any non-2xx status, or any failed above zero, as a failed upload, and gives up after 120 seconds without a response.

Through Gateway Hyper, the client gives up after 60 seconds with no bytes either way, so a batch the bundler is slow to accept fails there before publish's 120 seconds. The operator side answers 504 after 30 seconds of silence from the upload API by default.

The bundler's wallet signs and pays for the bundle, so the signer needs no AR and no Arweave wallet. Whether it accepts a data item from your signer is the bundler's own policy.

The signed data items aren't saved with the publish, so a publish cut off in uploadingViaBundler (or its second round) is marked interrupted when the page reloads, and the user publishes again. See Handle failures and resume.

User-paid storage: straight to Turbo

With payment: { storage: 'user' } in the publish config, the data items skip the upload API:

1. The publish package signs the data items as above. The signer's address is the account that pays.
2. It reads Turbo's free size limit (GET ${uploadUrl}/v1/info), prices the items above it (GET ${paymentUrl}/v1/price/bytes/{n}), and checks the signer's balance (GET ${paymentUrl}/v1/account/balance/ethereum?address=…). When the balance is short, it fails with InsufficientStorageCreditsError before uploading anything. With userStorage.paidBy set, it skips this check.
3. It POSTs each item to ${uploadUrl}/v1/tx/ethereum, with an x-paid-by header listing the paidBy addresses when there are any. Turbo charges those addresses' shared credits first, then the signer's. A 402 raises InsufficientStorageCreditsError.
4. On the multiPublish route it asks Turbo for each item's status (GET ${uploadUrl}/v1/tx/{id}/status) before attesting, and fails on a 404.
5. After attesting, the background worker finds each item's bundle through Arweave GraphQL (arweaveGraphqlUrl, then the default read gateways) instead of the upload API's status endpoint, and records the bundle's L1 ID as below.

uploadUrl and paymentUrl default to ArDrive's Turbo (https://upload.ardrive.io, https://payment.ardrive.io). Users buy Turbo credits for their signer's address with a card or crypto through Turbo. authorizePublish tokens aren't sent to Turbo. See Cost estimates and payment.

Html with embedded images

Html with data: images (under the default htmlEmbeddedDataUriPolicy: 'materialize') uploads in two rounds: first the images, then the Html rewritten to point at them. Each round is its own batch, and the bundler pays for both. A signDataItems callback is called once per round. Before upload, a publish writes the Arweave links in Html it hasn't published yet in the form htmlArweaveRefs sets. With the default 'gateway', links move onto the public gateway (arweaveDomain, arweave.net by default) and keep their path form, and the links that replace data: images and seed:property/ references are https://<arweaveDomain>/raw/<id>. With 'ar', they're all ar://<id>. See links in published content. What a publish does names the steps.

After attesting

The attestations record the data item ID. A background worker in the publish package (started by initPublish, in the leader tab) then polls GET /api/upload/arweave/status/{id} every 45 seconds. It reads the response as ArweaveUploadStatusResponse from @seedprotocol/arweave, and uses these fields:

The worker looks the bundle up first through arweaveGraphqlUrl and the default read gateways, then falls back to the bundleId the status gives. When l1.confirmed or l1.meetsMinConfirmations is true, it records the bundle's L1 ID in its arweave_l1_finalize_jobs table and stops polling that item. The local storage_transaction_id stays the attested data item ID, which is the ID gateways serve the file under.

Who signs what

None of these upload signers is the author. What ties a file to its author is the account's storage_transaction_id attestation. See Identity and signing.

Related

• Arweave and gateways: IDs, tags and verifying content
• Costs and who pays: what the upload API spends
• Gateway Hyper
• Overview and initPublish: uploadApiBaseUrl, dataItemSigner, signDataItems and the publish steps
• @seedprotocol/arweave: upload status helpers
• Publishing and revoking items