Skip to content

Instantly share code, notes, and snippets.

@kernelogic
Created June 18, 2024 16:12
Show Gist options
  • Save kernelogic/e1112cebfa793d466836b18dcb6fcc80 to your computer and use it in GitHub Desktop.
Save kernelogic/e1112cebfa793d466836b18dcb6fcc80 to your computer and use it in GitHub Desktop.

Kernelogic Fil+ Datacap Allocator

Background

  • Kernelogic as a notary, has allocated over 250 PiBs of datacap to clients, #1 on the leaderboard.
  • Kernelogic participated Space Race, Slingshot 1 and 2.
  • Kernelogic helped Fil+ team to analyze all AWS public datasets for Slingshot v3.
  • Kernelogic as a client, has onboarded over 100 PiBs of public dataset to the Filecoin network.

Workflow

In response to the new "Allocators and Meta-Pathway Design", Kernelogic is proposing its own Kernelogic Fil+ Datacap Allocator (KFDA).

KFDA will focus on verification automation and trustless DC allocation to eliminate as much human intervention as possible and maximize DC distribution efficiency.

KFDA will support both AWS public datasets and enterprise datasets as long as they are freely, publicly retrievable.

By collecting payments from clients, KFDA helps to achieve the ultimate "paying clients" goal of Filecoin.

Verification automation

  • For AWS public datasets, Kernelogic has already scanned all AWS open datasets and will store all the metadata on KFDA.
  • For other types of datasets, clients can upload dataset metadata to KFDA.
    • KFDA can set max DC needed according to metadata.
  • KFDA will rely on boost indexer and booster-http to sample and verify content of storage deals with KFDA metadata database.
    • Multiple verification server nodes will be set up in different regions to ensure higher retrieval success rate.
    • Verification details will be kept for audit purpose.
  • KFDA will implement local deal status database that similar to CID Checker bot to check for CID sharing, SP Geo location, distribution etc. Criteria TBD.

Trustless allocation

  • When client is ready for next tranche of datacap and data passed verification, client can pay Fil for datacap for a price per TiB. Then datacap is automatically issued to the client once the payment transaction is detected. Starting price 0.5F / TiB.
  • Various discount can be applied to the datacap price, such as
    • Dataset is a AWS public dataset, 5% off
    • Dataset is not overly crowded, 5% off
    • Client is working with SPs using green energy, 5% off
    • Client is working with new SPs, 5% off
    • Client is working with established SPs, 5% off
    • Client is working with SPs in certain region, 5% off
    • Client is working with SPs with higher retrieval speed, 5% off
    • Repeat client, 5% off
    • Therefore if client checks for all above, 40% off bring it down to 0.3F / TiB
    • More discounts in discovery

Application related questions

  • Who are the users?
    • Clients wish to onboard publicly retrievable dataset
    • SP enabled indexer and booster-http
  • How will the Notary ensure data alignment with their pathway?
    • Automated sampling and verification
    • Bot checker
  • What is the distribution methodology, and how will the Notary and Allocator pathway monitor deal distribution?
    • Automation: No manual verification
    • Permissionless: No entity requirement, no geo location requirement (but encouraged)
    • Trustless: No manual notary propose / approval steps
    • Network growth: 50 PiB per month DC allocated / onboarded goal
  • What is the budget or funding plan to support this allocation pathway?
    • Through Fil payment for datacap
    • Need 50 PiB initial allocated to KFDA
  • What tooling will be used?
    • filplus.storage for initial application intake
    • lotus binary to issue datacap
    • Lassie to verify data
  • How will the Notary bookkeep and handle disputes?
    • Audit trail is stored in KFDA database
    • If client failed verification for any reason, manual verification can be achieved to unblock.
  • How will the Notary iterate and adapt on the pathway over time? Are there specific change mechanisms that will be considered?
    • More ways to verify data
    • More discounts
    • Issue a utility token KFD on FVM to exchange for datacap. KFD could utilize Uniswap / Sushiswap to freely set KFD price after liquidity is enough.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment