voussoir
/
voussoir.net
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improve IPFS misconceptions. 

Network you're a part of:
- Get to the point sooner instead of leading with so much philosophical
  faff.
- Combine / remix paragraphs about billion-dollar entities.
- Improve wording about self-hosted web, game servers.
- Replace "website on cell phone" link to reddit with a screenshot
  demonstration.

Strangers host my files:
- Rename this section so that the names of all sections start
  with "IPFS".
- Replace blockquote from IPFS with inline quote for visual consistency.
- Greatly simplify the paragraph about strangers.
- Improve phrasing in general.
- Increase arrogance with the punchline at the end.

Anonymous:
- Combine some repetitive paragraphs.
- Simplify some phrasing.

IPFS homepage:
- Add link to specific revision of their page because it has
  since changed.
- Simplify some phrasing.
- Continue marvelling at the chadness of sqlite.
master
voussoir 2022-01-29 14:30:59 -08:00
parent fa9960e2f4
commit 16a4bfafc6
No known key found for this signature in database
GPG Key ID: 5F7554F8C26DACCB
1 changed files with 30 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

62
voussoir.net/writing/ipfs_misconceptions/ipfs_misconceptions.md
View File

@ -9,6 +9,8 @@ However, IPFS has marketing problems. I am subscribed to /r/ipfs which is inunda
## IPFS is not a place you upload to, it's a network you're a part of
All of the quotes in this article are real posts from /r/ipfs or other forums:
> Won't there be hard drive capacity isssues if everyone used IPFS?
> The way I understand it is everyone who opts in to be an IPFS server will download all the content on the web that I browse to, and host it.
@ -67,21 +69,23 @@ However, IPFS has marketing problems. I am subscribed to /r/ipfs which is inunda
The number one problem with IPFS's public perception is that a huge, huge number of people think that IPFS is a place you just "upload" to, and it'll hold your files forever. People see the word "decentralized" or "distributed" and assume that IPFS distributes your files for you, for free, everywhere.
This is understandable, since most end users are not accustomed to being an equal, active participant in the great glowing mass we call the internet. We are accustomed to interactions between service providers: Google, Apple, Facebook; and service users: us. We upload photos to Facebook and then leave. We upload videos to YouTube and then leave. The internet just runs 24/7 and I don't have to do anything about it.
IPFS is a peer-to-peer system, comparable to [bittorrent](https://en.wikipedia.org/wiki/BitTorrent). You are a peer, and you host files so that other peers can download them from you. When you use the "import file" button in the IPFS client GUI, or the `ipfs add` command on the CLI, you are simply telling the rest of the world that you have a file you'd like to share. It doesn't actually get uploaded anywhere until another person asks you for it. That's because IPFS is not a place you upload to, it's a network you're a part of.
Most of us feel that 'the internet' is this big thing that is separate from ourselves, managed by far away entities with billions of dollars, which we use and then leave; which we are not a part of. Proportionally, the number of people who have ever hosted a web server or game server themselves and had their friends connect, or transferred files between two devices in their home via LAN IP is dwindling all the time. Recently I explained to someone that a modem connects your house to the internet and your router connects your devices to each other -- and they asked me why you'd want to do that. The word "Internet" means "between networks", and necessarily postdates the smaller institutional- and home-level networks we started with.
Most end users are not accustomed to being an equal, active participant in the great glowing mass we call the internet. We feel that 'the internet' is this big thing that is separate from ourselves, managed by far away entities with billions of dollars. We are accustomed to interactions between service providers: Google, Apple, Facebook; and service users: us. We upload photos to Facebook and then leave. We upload videos to YouTube and then leave. The internet just runs 24/7 and I don't have to do anything about it.
You don't even need a domain name to have a website: just run an http server, open your ports, and give your friends your IP. You're live. Welcome to the internet. It's made by people, and you're a people too. You can even run a website [off of a cell phone](https://old.reddit.com/r/androiddev/comments/9lirv0/best_libraryframework_for_running_a_web_server_on/), except that Google and Apple are making this harder all the time in order to groom your Stockholm syndrome.
Proportionally, the number of people who have ever hosted a web server or game server for their friends, or transferred files between two devices in their home via LAN IP is dwindling all the time. Instead we rely on web hosting companies or "official" game servers and we forget that this power was ever within our grasp. Recently I explained to someone that a modem connects your house to the internet and your router connects your devices to each other -- and they asked me why you'd want to do that. The word "internet" means "between networks", and necessarily postdates the smaller institutional- and home-level networks we started with.
IPFS is not a place you just upload to and then leave. It is a network you're a part of. You need to download and run the client software, either on your home PC or on a rented server, and allow other people to connect to you. Then you need to *leave it running*, because if you turn it off then of course nobody can get your files. What did you expect.
You can be a part of the World Wide Web even if you don't own a domain name: just run an http server program, open your ports, and give your friends your IP. You're live. Welcome to the internet. It's made by people, and you're a people too. You can even run a website [off of a cell phone](phone_server.png), except that Google and Apple are making this harder all the time in order to groom your Stockholm syndrome.
To be a part of IPFS, you need to download and run the client software, either on your home PC or on a rented server, and allow other people to connect to you. Then you need to *leave it running*, because if you turn it off then of course nobody can get your files. What did you expect.
The selling point of IPFS -- the nature of its decentralization -- is that anybody who downloads your files can seed them to anyone else using the exact same link. The `ipfs://` links contain the hash of the file rather than the address of your computer specifically.
This is the difference between IPFS and the regular HTTP web: if a video gets deleted from youtube.com, it's *possible* someone else in the world made a copy, but how are you going to find that person? If you get lucky, someone will generously reupload it, but it's going to be at a different link. And if their link gets deleted, you need to find another one. Over and over again. With IPFS, the original link works as long as a single person in the solar system is willing to keep it alive.
This is the difference between IPFS and the regular HTTP web: if a video gets deleted from youtube.com, it's *possible* someone else in the world made a copy, but how are you going to find that person? If you get lucky, someone will generously reupload it, but the link will be different and hard to find. And if that link gets deleted, you need to find another one. Over and over again. With IPFS, the original link works as long as a single person in the solar system is willing to keep it alive.
There are companies or entities known as [pinning services](https://docs.ipfs.io/how-to/work-with-pinning-services/), who will host your files on IPFS for you if you pay them. However, even this is not "uploading to IPFS" any more than uploading a file to Amazon S3 is "uploading to The Internet". The agreement to host the file in exchange for money is between you and the service provider. IPFS the protocol and the IPFS community at large aren't responsible if that host goes out of business and stops hosting your file.
## "But why would strangers want to help me host my files?"
## IPFS does not force others to host your files
> Who pays for the storage on IPFS?
@ -97,7 +101,7 @@ There are companies or entities known as [pinning services](https://docs.ipfs.io
> If I run a node on my home server, how many terabytes do I reserve to replicate an infinitely expanding pin list, assuming I only have to host the content I offer or have previously downloaded?
> > What makes you think you can just "pin them all some place else as a free cloud backup"? Most pinning services charge you per GB*month (or equivalent) for the things you pin. If you stop paying, they stop pinning it.
> > What makes you think you can just "pin them all some place else as a free cloud backup"? Most pinning services charge you per GB\*month (or equivalent) for the things you pin. If you stop paying, they stop pinning it.
> > > I just want to be sure I understand the objective -- is it fundamentally a gimmick to part rubes from their crypto, or a genuine innovation in distributed storage?
@ -141,21 +145,19 @@ There are companies or entities known as [pinning services](https://docs.ipfs.io
This misconception often stems from the first, that IPFS is a giant magical hard drive in the sky where other people will hold on to your stuff for you forever, so naturally there must be some big incentive for them to do that. "What's in it for me?" is the only language we understand. That was the idea behind [Filecoin](https://filecoin.io/) -- it gives members a financial incentive to share their storage space with the network. As it happens, Filecoin operates on a [completely separate peer network](https://github.com/filecoin-project/specs/issues/1191#issuecomment-704065968), unreachable by the regular IPFS client, since they need to keep their accounts payable in order. Yet, people conflate Filecoin with IPFS itself, and assume that all IPFS involves the use of Filecoin or other payment, because they can't fathom what sort of incentives exist besides financial ones.
There is no "push" mechanism in IPFS. There is no forcing people to take a copy of your stuff if they didn't specifically ask for it. From the IPFS homepage:
> Each network node stores only content it is interested in.
There is no "push" mechanism in IPFS. There is no forcing people to take a copy of your stuff if they didn't specifically ask for it. The IPFS homepage says "each network node stores only content it is interested in".
That's right, idiots. Nobody's going to help you replicate your history book report for the same reason that they won't let you put your lawnmower in their garage after your garage gets full. They're not interested. This shouldn't be surprising.
This is related to the previous idea, that people are only familiar with online services backed by faceless corporations. It's difficult to imagine that the system is made of actual people and you have to make it worth their while to download what you're sharing.
This is related to the previous idea, that people are only familiar with online services backed by faceless corporations. It's difficult to imagine that the system is made of actual people and you have to make it worth their while to download what you're sharing. That's the magical effect of money, of course: people **will** suddenly become interested in your homework assignments and lawnmowers if you pay them to be. But in the absence of the financial incentive, you are left with the fact that nobody else is interested in your random junk files.
That's the magical effect of money, of course: people **will** suddenly become interested in your homework assignments and lawnmowers if you pay them to be. But in the absence of the financial incentive, you are left with the fact that nobody else is interested in your random junk files.
For many of these people, the line of reasoning stops there. If the network won't host my stuff for me for free, it's pointless. Ok, bye. Meanwhile, the people who generate valuable data that is interesting and desirable to others will see their files propagate. If NASA started advertising CIDs for original moon landing footage, there would be plenty of volunteers replicating it.
For many of these people, the line of reasoning stops there. If the network won't host my stuff for me for free, it's pointless. Ok, bye. Meanwhile, the people who generate valuable data that is interesting and desirable to others will see their files propagate. If NASA started advertising CIDs for original moon landing footage, they'd get plenty of help replicating it.
Many of these comments assume that all other IPFS nodes are strangers, but why? IPFS can very well be deployed in an organization where no one is a stranger to each other, and they don't need financial incentives to host each others' files. A website operator could serve their javascript or image assets over IPFS, and put lots of IPFS nodes around the world so that visitors can connect to the nearest one. If random strangers want to pin the assets, they can do that too, but the website owner would always know that keeping their files alive is their responsibility. They would never *rely* on strangers' pins when the functionality of their website is at stake.
Let me specifically address the point about "strangers". Because IPFS is free and global and public, any stranger can download a file you're hosting and become a second seed, and that's very kind of them. But IPFS as a protocol is agnostic to this. IPFS can very well be deployed in an enterprise setting where no one is a stranger to each other, and they don't need financial incentives to host each others' files because they're working for the same organization. A website operator like Google could serve their javascript or image assets over IPFS, and replace their traditional CDN servers with IPFS nodes that are programmed to replicate each other. This might sound pointless -- replacing one kind of server with another -- but it removes the necessity of DNS management and allows them to just dump IPFS nodes all over the place and let the protocol balance things automatically. If random strangers want to pin Google's assets, they can do that too, but Google would always know that keeping their files alive is their responsibility. They would never *rely* on strangers' pins when the functionality of their website is at stake.
Here is another example: imagine that a team of researchers in Antarctica (or on Mars) are accessing a website over IPFS. The first researcher will have to download the page and its assets over a slow satellite connection that leads back to the original node, but then they can seed the files to the other researchers over a faster local network. These peers are incentivized to help each other because it leads to a better experience for the whole group, not because they're getting paid.
Plus, if a team of researchers in Antarctica (or on Mars) are accessing google.com, only a single one of them needs to get the assets via satellite connection and then they can seed it to the rest of the camp over a faster local network. The point of IPFS is not "strangers keep my stuff", it's "people sharing files" for whatever reasons they need.
The point of IPFS is not "strangers keep my stuff", it's "people sharing files" for whatever reasons they need. You just have to look past your own nose.
## IPFS is not anonymous or "dark web"
@ -209,31 +211,29 @@ You can find lots of posts from people who are worried about illegal or unsavory
Some of these people believe that IPFS is a place you upload to, or that all nodes assist with the hosting of all files, which leads them to believe that by running an IPFS node they will wind up with illegal files on their computer. There is precedence for p2p networks where all peers help host all files, like [GNUnet](https://en.wikipedia.org/wiki/GNUnet) and [Freenet](https://en.wikipedia.org/wiki/Freenet), but this is not the case with IPFS. You only host what you visit and pin.
On the other hand, some people might just feel generally uneasy knowing that pedophiles can use new tools like IPFS to share their material, and they wish the system had some innate kind of block against that.
From a technical perspective, how do you expect the IPFS software to know whether a file is being blocked for a "good" reason, like preventing the spread of child pornography, versus a "bad" reason, like preventing the spread of Tiananmen Square photos? Both of these situations are basically the same: it's material that people are not allowed to look at. It just depends who's doing the allowing and disallowing in your jurisdiction. Should there be a human-curated ban list built into the default client? Curated by people from which country? The client is open source anyway so they'd just compile the program without that. It just doesn't work. Using technical solutions to address social problems rarely does.
On the other hand, some people might just feel generally uneasy knowing that child molesters can use new tools like IPFS to share their material, and they wish the system had some kind of innate block against that. But how do you expect the IPFS software to know whether a file is being blocked for a "good" reason, like preventing the spread of child pornography, versus a "bad" reason, like preventing the spread of Tiananmen Square photos? Both of these situations are basically the same: it's material that people are not allowed to look at. It just depends who's doing the allowing and disallowing in your jurisdiction. Should there be a human-curated ban list built into the default client? Curated by people from which country? The client is open source anyway, so criminals could compile the program without the blocklist. It just doesn't work. Using technical solutions to address social problems rarely does.
You'll also notice that knives do not have special handles that prevent you from committing murder, beer bottles do not prevent you from driving drunk, cash does not prevent you from buying drugs, pens do not prevent you from forging signatures. The tool is agnostic and the crime is up to you.
IPFS is nothing more than a protocol. It is a language that two computers speak when they talk to each other. There is no sequence of words in the English language that will instantly alert the police when I say them, no matter how bad the words are. Likewise, there is no kind of file that will cause IPFS to alert the police.
What stops people from distributing illegal files over IPFS? Nothing. IPFS is just a protocol. It is a language that two computers speak when they talk to each other. There is no sequence of words in the English language that will instantly alert the police when I say them, no matter how bad the words are. Likewise, there is no kind of file that will cause IPFS to alert the police.
However, (here comes the point of this section), IPFS is not anonymous. When you run an IPFS node and people connect to you, they're connecting to your public IP address assigned to you by your internet service provider. If the police catch wind of any CIDs for illegal files, they'll just check out which IP addresses are sharing that file and try to pin down the owners of those IPs.
However, (here comes the point of this section), IPFS is not anonymous. When you run an IPFS node and people connect to you, they're connecting to your public IP address assigned to you by your internet service provider. If the police catch wind of any CIDs for illegal files, they'll just check out which IP addresses are sharing those files and try to pin down the owners of the IP addresses.
People have begun to conflate decentralization with anonymity, which is not correct. As I described earlier, companies decentralize their CDN servers because it improves latency for each region, but they are certainly not anonymous. Tor, by contrast, aims for anonymity by never connecting two peers directly to each other, instead routing the conversation through several intermediate nodes.
People have begun to conflate decentralization with anonymity, which is not correct. As I described earlier, companies decentralize their CDN servers because it improves regional latency, but they are certainly not anonymous. Networks that aim for anonymity like [Tor](<https://en.wikipedia.org/wiki/Tor_(network)>) or [I2P](https://en.wikipedia.org/wiki/I2P) do so by routing your connection through several intermediate nodes so that the sender and receiver never talk directly to each other.
If a journalist in China tries to publicize the Tiananmen Square incident, the Chinese government might discover it and try to shut that person down. But, because a single `ipfs://` link can work anywhere in the world, that journalist only needs a single friend outside of the country to also pin the file and make it available globally, then the journalist can turn off their node. Other countries like USA will not prosecute a person for hosting the photos. (Realistically, the journalist should transfer the file to their friend through a separate, encrypted channel, not IPFS, in the first place.)
As with other technologies, like bittorrent or even just http, you could gain anonymity by running the program on a far away server which you rent (careful that your payment method doesn't reveal you), or by diligently routing your PC traffic through a VPN.
As with other technologies, like bittorrent or even just http, you could gain anonymity by running the program on a far away server which you rent or by diligently routing your PC traffic through a VPN. In both of those cases, you still need to make sure that the server owner or VPN service won't hand you over to the police and that your payment method doesn't lead back to your real name. The person who wants to download illegal files should prepare for the possibility of connecting to a honeypot source, and they probably should remove the file from their IPFS node afterward so it doesn't seed back to anyone else. This is already standard procedure.
When it comes to child pornography or pirated media, the IPFS situation is not much different than what we have now. If the hosting node is located in a region with lax laws, or there is no information linking the node to an individual, then there's no difference whether it's an IPFS node or a bittorrent client or emule or an HTTP website. The person who wants to download the files should prepare for the possibility of connecting to a honeypot source, and they probably should remove the file from their IPFS node afterward so it doesn't seed back to anyone else. Again, this is not much different from what we have now.
From what I've seen so far, the censorship-resistance claims of IPFS are predicated on the idea that you're first able to get your files out of the jurisdiction that wants to censor them and into a jurisdiction where they're permitted. IPFS does not provide any secrecy, and if the whole world is after you, IPFS isn't a magic shield. The fact of the matter is that the regular HTTP web already carries hate speech, copyrighted files, and illegal material, but new protocols are subjected to higher scrutiny and criticism than the incumbents as a matter of course.
From what I've seen so far, the censorship-resistance claims of IPFS are predicated on the idea that you're first able to get your files out of the jurisdiction that wants to censor them and into a jurisdiction where they're permitted. IPFS does not provide any more secrecy than bittorrent does, and if the whole world is after you, IPFS isn't a magic shield.
Now, one of those comments I quoted made a great point, which is that the reputation of the technology as a whole can be ruined if it's known for piracy or criminal material. When people give a demonstration of how to use a bittorrent client, they always show how to download a Linux release or Big Buck Bunny, because that's the only thing they can think of that isn't a copyright violation. Many people think that torrents are illegal in and of themselves. So far, IPFS hasn't become a popular piracy tool to my knowledge, and all of their messaging centers around webpages. The fact of the matter is that the regular HTTP web already carries both copyrighted and illegal material, but new protocols are subjected to higher scrutiny and criticism than the incumbents just as a matter of course.
Now, one of those comments I quoted made a great point, which is that the reputation of the technology as a whole can be ruined if it's known for piracy or criminal material. Bittorrent is known for little other than piracy -- when people give a demonstration of how to use a bittorrent client, they always show how to download a Linux release or Big Buck Bunny, because that's the only thing they can think of that isn't a copyright violation. Many people think that torrents are inherently illegal. So far, IPFS hasn't become a popular piracy tool to my knowledge, and all of their messaging centers around webpages.
## IPFS's copywriting is bad
Am I doing a good job so far? Let's compare what I've written with what appears on the IPFS homepage to see how poorly they communicate their project to newcomers. All of the statements they make are technically correct, but they only make sense if you already know how IPFS works. If you don't...
Am I doing a good job so far? Let's compare what I've written with what appears on the IPFS homepage to see how poorly they communicate their project to newcomers. All of the statements they make are technically correct, but they only make sense if you already know how IPFS works. If you don't, it's problematic.
**Note**: since this article was originally published, some phrasing on their homepage has changed. [This](ipfs_homepage_20210613.html) is the revision against which I wrote these criticisms. I will keep this article as-is until I take the time to review the new claims.
> **Today's web can't preserve humanity's history**
@ -271,15 +271,13 @@ Wording here could be changed to "when you add a file to your IPFS node" to remi
> IPFS removes duplications across the network.
This statement is poorly worded. I get it, because two identical files, or two files containing some identical blocks will produce matching hashes, so you don't get "duplicate" links and mostly-same files will sit better in the cache. But understanding the details of this sentence requires much more specificity than is given, so it is weirdly out of place.
In the context of "what happens when you add to IPFS", removing duplications sounds like the opposite of what the reader wants -- high availability and low latency is achieved only after many duplicates of the file are spread around the world and shared by many peers. By contrast, it is the centralized file hosts like Google Drive who benefit most from deduplication: if two people upload the same file to GD, Google can just store it once and let both people download it, saving themselves disk space.
This statement isn't specific enough. I get it, because two identical files will produce matching hashes, so you don't get "duplicate" links. But removing duplications sounds like the opposite of what the reader wants -- high availability and low latency are achieved only after many duplicates of the file are shared by many peers around the world. By contrast, it is the centralized file hosts like Google Drive who benefit most from file deduplication: if two people upload the same file to GD, Google can just store it once and let both people download it, saving themselves disk space. IPFS's statement almost lends itself to the interpretation that files will be removed from your node if they already exist on someone else's, which isn't the case.
Furthermore, this statement uses third-person voice for both IPFS and "the network", which is a double-whammy against the agency of the reader. It sounds like a remote service.
> Each network node stores only content it is interested in, plus some indexing information that helps figure out which node is storing what.
Should be reworded to "You only store content you're interested in" to remind that IPFS is not a remote service and give agency to the reader.
Could be reworded to "You only store content you're interested in" to remind that IPFS is not a remote service and give agency to the reader.
> When you look up a file to view or download, you're asking the network to find the nodes that are storing the content behind that file's hash.
@ -293,7 +291,7 @@ Should be reworded to "you're asking your peers to find who has a copy of that f
The IPFS copywriters should be aware that approximately 100% of today's content creators are accustomed to uploading their videos, music, photos, and blog posts to online ad-supported services without having to pay to do so. You probably should let them know that hosting their creations on IPFS requires them to run a node or pay a pinning service, and that the decentralization only happens after they have fans who are willing to store and pin their works.
IPFS's homepage is bad. I wonder if they are intentionally toeing the line with their messaging, since the implications are always more grandiose and fabulous than the reality. I've said it once and I'll say it again: sqlite.org has a front-page link to [When to use SQLite](https://sqlite.org/whentouse.html) which also plainly describes when **not** to use it. IPFS doesn't do this because they're [not as chad as SQLite](/writing/sqlite_what_a_hunk). Can't blame them on that one. No one is.
IPFS's homepage is bad. I wonder if they are intentionally toeing the line with their messaging, since the implications are always more grandiose and fabulous than the reality. I've said it once and I'll say it again: sqlite.org has a front-page link to [When to use SQLite](https://sqlite.org/whentouse.html) which also plainly describes when **not** to use it. IPFS doesn't do this because they're [not as chad as SQLite](/writing/sqlite_what_a_hunk). Can't fault them for that. No one is.
Clearly, IPFS's messaging, and the messaging from crypto nuts who advocate IPFS without really knowing what it is either, has led people to believe that IPFS is a giant decentralized cloud storage system that may or may not be free to use. In my opinion, the copywriters need to turn this ship around and make it clear that IPFS is not a magic dumping ground for your random personal junk to be replicated globally. It's a peer-to-peer network that you need to actually participate in if you ever want to get anything done.