The Martian Internet!

[Ultimate Steve]

So! Soon (ish) we may be headed toward Mars. And soon after that (ish) there may very well be a civilian population living on Mars. Provided they are there long enough, they would get pretty bored. Say you want to browse the KSP forums on Mars. Type in the homepage address. Wait 40 minutes. Click "Science and Spaceflight." Wait 40 minutes. Click this thread. Wait 40 minutes. Etcetera.

So, Mars at one point would most likely need its own internet. What would this sort of internet look like? How would it be different from Earth's internet? And when would this internet be made? When 1,000 people are on Mars? 100,000? 1 million? If you could start fresh with the internet, what would you do?

Discuss. Also, if starting topics count as posts, then this is my 400th post.

 

[stibbons]

The local network would look exactly the same as it does here. Although I hope we've stopped using IPv4 on most of the public Internet since then. 

The interplanetary link is interesting - TCP/IP wasn't designed to deal with the long delays involved, and NASA and others have started researching how to augment existing protocols to deal with it. Looking forward to the day space agencies standardise on using nearly identical protocols to the rest of the world for their comms. 

But you seem to only be concerned about the application layer, so:

It'll mostly come down to frontend sharding, with UI enhancements to better communicate to users when content was published. To use Facebook as an example, they'll have a local Web presence everywhere that matters, and all of the long-range content syncing will happen behind the scenes. But I'd expect the user experience to more accurately convey that there's a significant difference between when somebody posted something and when you received it. 

As for your specific example, I hope it would finally be the death of Web forum software. There's literally nothing a forum does that isn't better handled by email and/or usenet, with the added benefit of better threading and handling of asynchronous comms. Boo to webifying everything. 

[Nibb31]

3 hours ago, Ultimate Steve said:

So! Soon (ish) we may be headed toward Mars. And soon after that (ish) there may very well be a civilian population living on Mars.

We headed to the Moon in the 60's and there is no civilian population on the Moon.

There won't be Mars colonies before centuries at least.

Quote

So, Mars at one point would most likely need its own internet. What would this sort of internet look like? How would it be different from Earth's internet? And when would this internet be made? When 1,000 people are on Mars? 100,000? 1 million? If you could start fresh with the internet, what would you do?

There are so many more harder  problems to solve before we get to that point than how to use the Internet.

I think you would likely see the return of asynchronous protocols like email and usenet and use local caching for some of the more popular websites (like Wikipedia, Netflix, news sites, Youtube, etc...). It certainly isn't dealbreaker.

Quote

Discuss.

Not if I don't want to.

Edited August 21, 2016 by Nibb31

[Shpaget]

Assuming you have the bandwidth, it can be done without the 40 minute ping. You just cache the heck out of it. You basically make a copy of the most visited sites and periodically sync it with Earth.

With forums it's not too hard, since the amount of data transfer is low and the communication isn't realtime anyway, so it doesn't really matter if the post you read was posted moments ago or a few hours ago. While clicking and browsing the forum you would be browsing the local version that is a few hours old. From the users perspective it would be identical to current access, with the only difference being the lack of some posts that were made since the last sync. When the user writes a new post, it is stored in the Martian version and when the next sync comes up, the Mars and Earth versions are combined. It wouldn't be seamless, since some topics get a lot of comments in a short amount of time, but for general participation it would work fine.

While I'm not sure how would such a data link be achievable, I imagine it would require some huge antennas and a sizable power supply, but if we had a large population on Mars, it would certainly be one of the projects on the To Do List.

This article talks about the efforts to achieve 600 Mb/s between Mars and Earth using existing and currently deployed equipment. I imagine that a dedicated broadband relay system would be significantly faster.

[Laie]

2 hours ago, Shpaget said:

Assuming you have the bandwidth, it can be done without the 40 minute ping. You just cache the heck out of it. You basically make a copy of the most visited sites and periodically sync it with Earth.

Basically, this. I'm still old enough to remember UUCP-to-FIDO gateways and expect that (some of) the same principles would apply.

Simple: Person a Mars requests a Wikipedia page; proxy on earth retrieves it and the whole bundle is transferred to a proxy on Mars.

Fancy: you also fetch every page the original article links to, and follow every link on these pages, to an arbitrary depth. Some "smart" algorithm may determine which links are more likely to be followed so the request doesn't balloon quite as madly.

Fancy2: proxy monitors the more favorite sites on earth and pushes changes immediately, without waiting for a request from Mars.

 

[Tex_NL]

2 minutes ago, Laie said:

Basically, this. I'm still old enough to remember UUCP-to-FIDO gateways and expect that (some of) the same principles would apply.

Simple: Person a Mars requests a Wikipedia page; proxy on earth retrieves it and the whole bundle is transferred to a proxy on Mars.

Fancy: you also fetch every page the original article links to, and follow every link on these pages, to an arbitrary depth. Some "smart" algorithm may determine which links are more likely to be followed so the request doesn't balloon quite as madly.

Fancy2: proxy monitors the more favorite sites on earth and pushes changes immediately, without waiting for a request from Mars.

 

Without some sort of fancy sci-Fi FTL communication this is pretty much the only way to avoid the half hour lag between request and reply. Unless you can mirror every website you will need some sort of basic AI to predict what is going to be needed and cache it in advance.

[magnemoe]

1 hour ago, Tex_NL said:

Without some sort of fancy sci-Fi FTL communication this is pretty much the only way to avoid the half hour lag between request and reply. Unless you can mirror every website you will need some sort of basic AI to predict what is going to be needed and cache it in advance.

Yes, systems like this is in use today however their main focus is reducing bandwidth demands and server load as speed is not an issue outside of real time stuff like remote control or games.
An major issue will be scripted stuff, this is most sites however interactive ones like this forum will have to be replicated. 

[StrandedonEarth]

That poor guy on Duna Mars will get ninja'd to the dogs and back

[kerbiloid]

They can store a copy of internet on diskettes in the Martian data center and exchange with changes (diffs) on schedule. An incremental partial update.

The Martian colony is not a question of the nearest future. So:

Personal harddrives are replaced with cloud storage accounts for 99.99% of people.
Nerds bristle, scream, make their personal data storages - doesn't matter, as 99.99% of internet are movies, cats, selfies and random posts.

FaceGoogle & MicroApple Corp. builds a titanic data center with an internet copy. Including almost all personal files in the world: i.e. what was on all people's harddrives a decade before.
All files are unique, with unique checksums, no duplicates.
So, rather than on your personal harddrive, you own not files (including 20 copies of the same valentinas, 3 copies of the same cat photo, 1 copy of the same movie as all your neighbours have too), but just access permissions.
So, 1 copy of a movie is shared between a million of people which means 2 GB instead of 2 000 000 GB. Total data volume dramatically decreases.

They build twenty more same data centers around the world. They are replicas and contain the same.

As all sites arre now hosted in one place, in one huge database, they get unduplicated..All texts get indexed and stored as diffs. 
So, 1000 senseless posts about vacation become 1 base post with 10 unique diff versions (as some of them, say, use "girlz" instead of "girls") and 20 lolmaps. Nobody even notice a difference.

When a Martian colony appears, they just build a Martian replica of data center and fill it once with a diskette snapshot.
Earth and Mars data centers continue mirroring each other in background.
Most of text diffs are predicted by a neural network. Unborn posts from Earthlings are being semi-randomly created before they will be required and are stored as:
"Predicted post 32650856. TemplatePhrase #128743847. Diff:  person_name replaced with Jim. Lolmap # 45. Status: awaits confirmation."

So, all looks nice.

2020. Siri writes forum posts as told.
2025. Siri asks you what to write.
2030. Siri suggests you what to write.
2035. Siri informs you what she had posted.
2040. You know that Siri herself knows better what to post.
2045. Siri predicts what you will need and your get a search response before you ask an answer.
2050. Siri predicts what you will be answered you get an answer from other people before you send them a question.
2055. Siri has all what you need.
2060. You're an avatar of Siri.

[magnemoe]

1 hour ago, kerbiloid said:

They can store a copy of internet on diskettes in the Martian data center and exchange with changes (diffs) on schedule. An incremental partial update.

The Martian colony is not a question of the nearest future. So:

Personal harddrives are replaced with cloud storage accounts for 99.99% of people.
Nerds bristle, scream, make their personal data storages - doesn't matter, as 99.99% of internet are movies, cats, selfies and random posts.

FaceGoogle & MicroApple Corp. builds a titanic data center with an internet copy. Including almost all personal files in the world: i.e. what was on all people's harddrives a decade before.
All files are unique, with unique checksums, no duplicates.
So, rather than on your personal harddrive, you own not files (including 20 copies of the same valentinas, 3 copies of the same cat photo, 1 copy of the same movie as all your neighbours have too), but just access permissions.
So, 1 copy of a movie is shared between a million of people which means 2 GB instead of 2 000 000 GB. Total data volume dramatically decreases.

They build twenty more same data centers around the world. They are replicas and contain the same.

As all sites arre now hosted in one place, in one huge database, they get unduplicated..All texts get indexed and stored as diffs. 
So, 1000 senseless posts about vacation become 1 base post with 10 unique diff versions (as some of them, say, use "girlz" instead of "girls") and 20 lolmaps. Nobody even notice a difference.

When a Martian colony appears, they just build a Martian replica of data center and fill it once with a diskette snapshot.
Earth and Mars data centers continue mirroring each other in background.
Most of text diffs are predicted by a neural network. Unborn posts from Earthlings are being semi-randomly created before they will be required and are stored as:
"Predicted post 32650856. TemplatePhrase #128743847. Diff:  person_name replaced with Jim. Lolmap # 45. Status: awaits confirmation."

So, all looks nice.

2020. Siri writes forum posts as told.
2025. Siri asks you what to write.
2030. Siri suggests you what to write.
2035. Siri informs you what she had posted.
2040. You know that Siri herself knows better what to post.
2045. Siri predicts what you will need and your get a search response before you ask an answer.
2050. Siri predicts what you will be answered you get an answer from other people before you send them a question.
2055. Siri has all what you need.
2060. You're an avatar of Siri.

2065 people stops using digital communication but leaves this to the computers. 
2070 the drift between the avatars and actual people increases because of the lack of communication. 
2075 digital communication end up like online gaming 40 years before, an computer only arena.
2080 the martian server park to earth for faster communication between the computers 

More serious the indexing of files is pretty common for large data storage and backup.
I know that an controversy with the Megaupload file sharing service was that, they removed links to copyrighted material they was ordered too, however this did not remove the file and other links to the file was not removed. 

 

[kerbiloid]

2 hours ago, magnemoe said:

More serious the indexing of files is pretty common for large data storage and backup.
I know that an controversy with the Megaupload file sharing service was that, they removed links to copyrighted material they was ordered too, however this did not remove the file and other links to the file was not removed

A commonly used object reference counter, speaking programmatically.
While an object is in use and referenced by some variables, its reference counter is >0.
When a link gets deleted, the object stays intact, but its reference counter gets decreased by one.
Periodically a garbage collector process loops throw the objects list and removes those whose counters <= 0.
Why to keep several absolutely similar letters from Mary to John and from Ann to Jim when we can keep their 99% common text, and several words to be replaced.

[jwenting]

problem with caching is that a lot of internet traffic (potentially the majority) isn't static content. It's dynamically generated as needed and on the fly based on user requests.

While you can cache wikipedia and all you get is a delay in updates equal to the cache refresh interval, your banking website (just one example) can't be cached because it changes on the fly and for every user who logs in.

While a lot of that data might not be relevant over interplanetary distances, a lot of it is (news services also work like that for example).

And a LOT of internet traffic isn't websites at all, it's business data flowing over the net between companies, government agencies, and individuals. Think emails, VOIP calls, teleconferencing, VNC sessions, web service calls.

[KAL 9000]

I think that there will be a duplicate of the Internet on Mars. All websites and stuff are there, and communicating with someone else on Mars will be the same as the Internet today. Then, let's say that we all move to Mars and Scott Manley stays on Earth. That is bad. So, every hour or so, Mars sends all web updates to Earth, and vice versa. Both planets update their servers with the new data.

[kerbiloid]

Keyword is: redundancy.
It allows to zip several terabytes of raw video into 2 gigabyte mediafile. It allows to undrstan incmplet sentencs.
And what you see on a screen is a prediction of intermediate pixel states based on several hundred key frames.
As most of people are not so original to want something really unusual, almost all of they can need can be predicted and pre-calculated or pre-uploaded,

The main problem are binary question, yes/no. Most of other answers can be emulated with an appropriate precision, especially when both talking persons are googling same pictures while discussing the same movie. As Elisa program has demonstrated , anyway most of people's dialogs are monologs.

Really important information about person's life can be delivered in background and just be shown to the targeted person before or during the skype conversation (livefeed events in sidebar), preventing the need to ask this.

1 hour ago, jwenting said:

And a LOT of internet traffic isn't websites at all, it's business data flowing over the net between companies, government agencies

By only puny remains of it is shown to the person watching a business terminal.

1 hour ago, jwenting said:

emails, VOIP calls, teleconferencing

When you every minute have a livefeed before your eyes, with "Mary has married John. Jim is now having breakfast and likes a post of Ann..." how often would you mail/skype them to ask about these events?

Edited August 23, 2016 by kerbiloid

[Slam_Jones]

On 8/21/2016 at 0:53 AM, Nibb31 said:

-snip-

Not if I don't want to.

So, forgive me for asking, but why even post then?

[magnemoe]

23 hours ago, jwenting said:

problem with caching is that a lot of internet traffic (potentially the majority) isn't static content. It's dynamically generated as needed and on the fly based on user requests.

While you can cache wikipedia and all you get is a delay in updates equal to the cache refresh interval, your banking website (just one example) can't be cached because it changes on the fly and for every user who logs in.

While a lot of that data might not be relevant over interplanetary distances, a lot of it is (news services also work like that for example).

And a LOT of internet traffic isn't websites at all, it's business data flowing over the net between companies, government agencies, and individuals. Think emails, VOIP calls, teleconferencing, VNC sessions, web service calls.

Many heavy visited pages like major newspapers uses internal caching to lessen the load on the content servers. 
Yes its not the same, they also often caches only part of the page so it can add stuff like adds or adjusted content in addition.

For mars it would be relevant to cache everything you load, if you have live content like facebook and you leave the page its better to get the old version than having to wait 20 minutes for it to open. You might have an timer for next update. 
The most relevant and important pages like vikipedia would be cached and updated regularly, its also fairly static content if its an day old it hardly matter.

 

[WillThe84th]

They could have some way of constantly downloading information form Earth, so you wouldn't have to wait 40 minutes for a page to load, but you would have to wait forty minutes for any new information to come across.

[jwenting]

12 hours ago, magnemoe said:

Many heavy visited pages like major newspapers uses internal caching to lessen the load on the content servers. 
Yes its not the same, they also often caches only part of the page so it can add stuff like adds or adjusted content in addition.
 

I know how it works, I write such systems for a living

And we also ALWAYS include no-cache directives to prevent proxies and clients from doing their own caching to prevent them getting out of date content...

But as I stated, most traffic isn't websites at all. It's data flowing between computers that drives other applications. Your VOIP calls are but one thing (and what most consumers would recognise), and your bank data. But if you're a stock trader you get data as well, lots of it, and having cached data isn't going to cut it. Of course inter-planetary stock trading isn't going to be very lucrative. Far better to hire an agent on the other planet to do your trading for you based on certain guidelines and rules.

[kerbiloid]

10 minutes ago, jwenting said:

But if you're a stock trader you get data as well, lots of it, and having cached data isn't going to cut it

Do the Earth traders have any breaks? Say, night or weekend?

P.S.
When somebody exchanges with forum posts, they are not even sure if the conversation partner would response a minute or a year later.
But billion of humans still do this. So, voip and so could be partially replaced with videoposts exchange if necessary.

Edited August 25, 2016 by kerbiloid

[Gordon Fecyk]

On ‎8‎/‎23‎/‎2016 at 7:51 AM, kerbiloid said:

So, rather than on your personal harddrive, you own not files (including 20 copies of the same valentinas, 3 copies of the same cat photo, 1 copy of the same movie as all your neighbours have too), but just access permissions.
So, 1 copy of a movie is shared between a million of people which means 2 GB instead of 2 000 000 GB. Total data volume dramatically decreases.

I deal with this kind of tech daily. IT wonks call it 'deduplication.'

I saw it first on Backup Exec for disk-based backups and later saw it on Windows Server 2012 and 2012 R2 for normal file volumes. Only these go beyond deduplicating files: They deduplicate file clusters on disk, doing 'block deduplication.' It's pretty funny seeing a file with these attributes: Size 419 KB, Size on Disk: 4 KB.

You can bet everyone offering cloud storage today is using some form of block deduplication. I imagine caching servers on Mars would save a ton of bandwidth and storage doing this when syncing with Earth servers, and the same thing would happen in reverse when Martian sites and services become popular back home... come to think of it, Backup Exec already does this for offsite copies of deduplication stores, calling it "optimized deduplication."

--

[juanml82]

On 23/8/2016 at 2:17 PM, jwenting said:

problem with caching is that a lot of internet traffic (potentially the majority) isn't static content. It's dynamically generated as needed and on the fly based on user requests.

While you can cache wikipedia and all you get is a delay in updates equal to the cache refresh interval, your banking website (just one example) can't be cached because it changes on the fly and for every user who logs in.

While a lot of that data might not be relevant over interplanetary distances, a lot of it is (news services also work like that for example).

And a LOT of internet traffic isn't websites at all, it's business data flowing over the net between companies, government agencies, and individuals. Think emails, VOIP calls, teleconferencing, VNC sessions, web service calls.

It would be centuries before interplanetary corporations are bothered by not having instant access to their bank records on Earth while on Martian low orbit, though. We don't even know what the Internet will be that far in the future.

I'm not even exactly sure interplanetary corporations as opposed to franchises operated by separate companies are a viable concept until there are tens of thousands of people permanently living in Mars. After all, it's not like there are bank offices or Starbucks in Antarctica, and people have been living there, in a temporary capacity, for decades.

[Laie]

 

On 8/23/2016 at 7:17 PM, jwenting said:

And a LOT of internet traffic isn't websites at all, it's business data flowing over the net between companies, government agencies, and individuals. Think emails, VOIP calls, teleconferencing, VNC sessions, web service calls.

That's missing the point, in a way. Barring FTL communications, low-latency interactive services just won't work. There's no reason why they should make up the bulk of the traffic. You can cache around the latency when it's non-interactive, but assuming a limited cache size, this won't always work. People have to deal with it. Likewise they will (have to) find alternative ways to get their stuff done (like mail instead of IM).

Heck, this goes way down the line: However data is exchanged over the link, you can't wait for an ACK every 1500 bytes.

On 8/25/2016 at 7:21 AM, jwenting said:

Far better to hire an agent on the other planet to do your trading for you based on certain guidelines and rules.

Yep. that's more like it.

[jwenting]

On 25-8-2016 at 7:28 AM, kerbiloid said:

Do the Earth traders have any breaks? Say, night or weekend?

Nope, trading goes on 24/7, 365 days a year. It's in large part automated, and traders work in shifts to monitor things.

[kerbiloid]

9 hours ago, jwenting said:

Nope, trading goes on 24/7, 365 days a year. It's in large part automated, and traders work in shifts to monitor things.

Then Earth-Mars traders should implement new type of contracts: "delayed light futures" for the cases when they can't delegate their trading for an exterritorrial comissioner.

(Where "delayed", "light" and "future" are mentioned literally).

Edited September 2, 2016 by kerbiloid

[grawl]

These fast trading needs might drive investment for the development of entangled particles based communication protocols.

I wonder if those could be faster than light ? Virtually instantaneous ?