Crypto AMA with Arweave (11.8.19)

Please welcome Sam Williams of Arweave to Crypto AMA!

As a reminder for everyone participating—please keep the discussion respectful and on-topic at all times.

@Sam — could you start off by giving us a brief bio on your background as well as how you got started in crypto? And then a short overview of Arweave and a brief update on your progress to date? We’ll then be off to the races with questions.

Sure!

Firstly, thanks for the invite. Excited to chat to you all.

I was always fascinated by distributed operating systems and networks (particularly Bittorrent), but I first got into crypto (from a spectators point of view) somewhere before Bitcoin gained parity with the US dollar. At that time I didn't think that it was much more than an interesting technical experiment. Later, I mined a bunch of Bitcoin on my PC in my university dorm ('free' electricity!), which I put into the Ethereum ICO as soon as I had read the promise of 'programming a world computer'.

The 'world computer' didn't really materialise as I had hoped. Nonetheless, in the meantime I had begun a PhD in decentralised operating system design. The OS I built was an attempt at using decentralisation techniques inside a single computer to achieve fault tolerance of hardware errors during operation.

During the final year of my PhD, I started thinking pretty carefully about the problems associated with disinformation on the web (this was shortly after the 2016 US presidential election). The web has rapidly become the tool that we use to communicate and come to consensus as societies, yet there is no strong identity, immutability, or 'audit' trail of who said what, when. It was clear to us that blockchains have the a number of properties that would be useful on the web (immutability, strong identity, decentralised ownership/maintenance etc.), but could not scale to store large amounts of data.

That is essentially why we started building the Arweave -- to create a permanent, decentralised web that maintains the properties of a blockchain.

We spent a few months tinkering with ideas for how to achieve the necessary scaling mechanics, until we came up with the blockweave and proof of access structures. After that, we built a prototype of the network, a paper describing the basic techniques, and raised ~$400,000 in pre-sales of the networks token. This was enough to take us through from prototype to fully audited network launch, 10 months later.

That was June 2018. Since then we have been focusing our attention on network adoption and improvements. We have recently hit our stride on the adoption front, and the 1.x branch of the protocol has reached maturity. Most months we hit >50% MoM network storage growth, and we have gone from 1 community built app on the network to ~150 in just 5 months or so.

Ok, essay over. Now ask me anything 😅.

Great introduction. Thanks @Sam.

Let's jump into questions!

How does the Arweave data structure differ from a blockchain?

Hi Sam - thanks for the overview. I love what you guys are building technically and it's a very important piece of infrastructure.

How do you think about the concrete use cases that will make Arweave a sustainable ecosystem? I've heard ideas around archiving journalism or public documents, both of which are very important for the world but not really massive business opportunities or markets. Whereas if you had even 100 large enterprises building on this it could be a very large business and ecosystem.

Do you have thoughts on the commercial use cases that will make Arweave work?

Thanks for the time, @Sam.

Hi Guys!

This may be off topic, but It seems arweave need a catalyst, maybe like the permanent storage of some file or data that’s useful for a wide range of people to really drum up interest. Do you have any ideas on what this catalyst could be or if you’re working on one?

We use a blockweave data structure, rather than a blockchain. The essential difference is this: we include the entire contents of a random previous block in the production of the next block, as well as the hash of the previous block. This enforces that miners prove that that they have access to a random sample of old data in order to start the computational component of mining. Access to more of the dataset -> higher likelihood of being able to hash. Details in the yellow paper on the site.

This

and this

Seem related. So if you don't mind, I will answer both at once:

Very helpful. Thank you, Sam!

How can Arweave complement the Internet Archive?

Can you describe Arweave’s token economic model?

Could Arweave be extended to support smart-contract-like computation, or do you see that as an entirely distinct concept? E.g. running a script and storing the script, inputs, and and outputs. I’m picturing something where the computation may require a lot of memory.

Permanent storage is a utility that you just simply could not credibly purchase previously. This is great, because it opens doors to the development of many applications that could not have been built previously. But it is also brings complexity from a strategic point of view: where is the right beachhead to attack first? Further, we are not domain experts in all of the areas that info permanence will be valuable -- so how can we effectively execute on these potential verticals?

Our solution to these problems is grass roots:

1. Focus on attracting developers to the ecosystem that have a wide variety of backgrounds.

2. These developers, in turn, then build applications on the network that provide domain specific user interfaces for various different industries in which they are experts (legal tech, finance, consumer applications, etc.).

3. We then pick the best of these apps and attempt to promote them in the appropriate arenas.

Step 1 of this is well underway, as is step 2. Mechanisms for step 3 are in the works (incentivising marketers to promote the apps, startup grants/pre-seed funding, etc.). Members of our dev community are already organically pitching Arweave to their organisations and friends. One for example, works at a 'big four' auditing firm and has been advocating for Arweave integration into their processes.

The first app on the Arweave was 'CRIAG' -- the Censorship Resistance Internet Archive Gateway. It archived torrents from the IA, then made access to them available via Arweave. This gives strong time stamping and distributed access to IA data at an exceptionally low cost, but does not ensure the permanence of the IA dataset itself.

Imagine Bitcoin (you pay Sats for every byte in your TX), except that when you submit a TX, the fee goes into an endowment pool, from which it is released overtime, rather than being given immediately to the miner.

The endowment gains 'interest' on the 'principle' that you pay upfront in the form of declining storage costs. The network assumes a base expectation of 0.5% annual storage cost decline (the average over the last 50 years has been 30.5%). As long as the storage cost decay rate stays above this amount, the payment for storage will be perpetual, from your single upfront fee. Even if somehow it dropped below that rate, your upfront fee would cover literally hundreds of years worth of storage fees.

The cost of setting up the endowment is approximately $0.005-$0.01/MB.

Lots more details about this in the yellow paper!

http://arweave.org/yellow-paper.pdf

We have optimised Arweave for data storage. We like the old Unix mantra: do one thing, and do it well.

Of the 150 apps now on the network which are getting the most usage?

Arweave does, however, integrate trivially with all of the existing smart contract systems.

@ryan and I have also been throwing around ideas for how a 'lazy' second layer smart contract system could be built on top of Arweave, but that is a conversation for another day.

The Arweave web extension (which allows you to archive web pages from the centralised web) is the most used right now.

Thanks, any others you follow closely?

What are the main differences between Arweave and Filecoin?

You shared the RandomX audit with Monero, how has it gone with Arweave? 

Any thoughts on it going live in XMR at the end of the month?

I am really excited by the decentralised autonomous web apps (DAWAs) on the network. They look like normal web apps, but underneath they are completely different: They are owned and controlled by nobody, except the code itself. This means that their moderation policies and behaviour are programmatic in nature, rather than mediated by a human.

There is a huge amount to say about the implications of this, but this doesn't seem like the right forum. Happy to chat about it after.

An example would be one of the many forums that live on the network: https://5vbgactgfkjm.arweave.net/3SMZ1RW2xGTBdTiUznKrkTJTHtYtBs27wm0q2H58Jck

Really cool stuff, thanks!

No signs of ASICs or GPU implementations yet! So far, it appears the 'ASIC-resistance' claim holds up. The logic for the claim is very strong, but it will clearly be quickly put to the test when Monero switches over.

Very little. Filecoin is going for a Sia-style temporary storage system, where the valuable proposition comes from theoretically lower costs. The logic is that storage can be provided cheaper in a decentralised network than Amazon can in a highly optimised centralised setup, although I am not sure that I buy this.

Arweave takes a totally different approach. The network offers a utility you just cannot buy in a centralised market. It  is the only place that you can purchase truly _permanent_ information storage -- a completely separate value proposition.

Gotcha, makes sense.

What are some of the potential Byzantine actions against the Arweave network that keep you up at night?

Arweave is essentially structured like Bitcoin, from the point of view of attack vectors. This makes the main vector the classic '51% attack' (although the potential effects of this are often misunderstood in the wider crypto sphere).

One of the surprising effects of not yet being listed on a major exchange has been that competition for mining has been fierce -- alleviating much of the concern around >50% attacks during the 'bootstrap mining' phase when hash rate is typically low. Now, given the recent funding announcement with a16z, USV, and Multicoin, we are further seeing this effect compound -- more miners hunting for access to AR.

To be honest, what really keeps me up at night recently is seizing and driving the momentum that we have on the adoption front. When we started on this mission to build 'a new web', it seemed like an unbelievably distant goal. Now, 2.5 years later, it seems an awful lot more achievable.

Every second counts 😅.

Good stuff, thanks @Sam.

15 minutes, everyone - get your questions in!

What was the motivation for using proof of work?

Also, can you describe the different facets of Arweave governance?

We don't really, we use proof of access, which contains a proof of work component. The interesting thing about proof of access is that as more data is stored, less computational power is expended in the network. Hashing essentially gives way to storage over time.

Details in the paper!

What are your thoughts on using blockchains as a data availability layer? Do you think arweave can play a role in that?

We kept consensus layer governance light when designing the protocol, aside for the case of content moderation. 

The reason for this is that unfortunately, no matter what governance procedures one puts on-chain, we cannot escape the 'base truth' that the miners will run whatever code they like -- causing chain forks if they desire it. No amount of on-chain governance will be able to escape that fact.

Nonetheless, the protocol supports 'tagging' blocks and transactions with metadata. We have long planned a small permaweb app that uses this mechanism to allow miners and archivers to signal support/dissent for network upgrade proposals. Given the 'ground truth' of on-chain governance procedures, this is functionally equivalent to the more complex schemes.

_Blockchains_ aren't a great option for this, because their storage doesn't scale and is expensive. Arweave solves these problems, however, so it seems like a great use case!

(Not that I am biased at all 😉)

What are your thoughts on something like lazy ledger

https://arxiv.org/abs/1905.09274

Guys, unfortunately we have a hard stop at 2:30pm so we've gotta end this.

@Sam thanks a ton for coming on today! What's the best way for people to get in touch and stay apprised of Arweave's progress?

Great question, which I am afraid I don't have time to answer!

Thanks for having me!

Twitter is the best way to keep up with the team: https://twitter.com/arweaveteam

And you can reach me on weavemail.app at: vLRHFqCw1uHu75xqB4fCDW-QxpkpJxBtFD9g4QYUbfw