« Forecast for next few months: bugs and scaling | Main | Straight on 'til morning... 3,000 hours in NeverLand »

October 30, 2004


Morgaine Dinova

In reference to:

"SL of course makes no set distinction between geography - we can all go
wherever we like - it is our mid-term goal to deploy servers that are
'close' in terms of network delay to the folks most heavily using them.
The distributed nature of SL's technology enables this. So it is
quite likely that as SL grows, there will be places in-world that at
least loosely map to the actual places on earth where the servers are

Philip, this idea of moving servers to closer geographic proximity of subscribers is ill-founded and will impact on your ability to scale, except in the sole case of caches. The issue are nothing like those in geographic distribution of web servers or of MMOG universes where a player's server set does not interact with others.

In doing so, you are somewhat hardwiring in the concept of static server allocation, and making it even worse by tying servers to zones to 1L geography. You reduce your ability to aggregate servers into dynamic pools, because a box in a European cluster will never be able to be aggregated dynamically into a US cluster. (It can be done, but the performance is poor.)

Even worse though, it misses the key point that where you need most server power is at events, not in your home zone (except coincidentally). Geographic proximity does not help at all unless you are trying to promote regionalized separation of events and nationalism, which would be so utterly appalling that I cannot believe it to be the case.

I would examine this idea more closely before you put it into effect. At some point SL will grow so huge that the object-serving pools will need to be split into sub-pools with streaming caches between them, and at that point you might as well consider geographic separation, but that is a long way off.

Morgaine Dinova

I mentioned regionalized caches, but said nothing about them, so here goes.

If a system employs caching servers, it always makes sense to regionalize them when there are long-lived objects in the world that can be cached and when the client experience is strongly RTT-sensitive. That follows from the standard benefits of caches, namely reduced loading on the master servers and faster perceived response to the client. Ie. it goes without saying. :-)

Currently the client has a problem in that a single thread is used for both rendering and networking, which couples the two tasks and must affect both very adversely. No doubt this is being addressed, since Lindens have mentioned this on the forums. Even then however, client-server RTTs will continue to affect player experience very strongly, and therefore regionalized object caches always make sense.

Unfortunately, there is a related issue that would totally undermine the ability of regionalized caches to do their job well, just as it undermines the ability of client caches to do their jobs well right now. That issue is the "dirtying" of objects (from the point of view of invalidating cache entries) by scripts, and the contamination of this dirty status to the entire link set of which the object with the script is a part.

Because SL is so strongly script-oriented (and will probably become even more so as LSL improves), the above problem very strongly impacts on the good job that the client cache is trying to perform, and exactly the same would apply to regionalized caching clusters.

Regionalizing caches aside, algorithmic changes are needed in this area. Object dirtying with scripts needs to be reduced in its effect on cached object invalidation, and link set contamination needs to be stopped entirely except when other objects in the set are actually affected.

Morgaine Dinova

From the SL Forums->Technical Issues, "Caches create a more level playing field."

In a nutshell, geographic regionalization of object caching clusters could help everyone in principle, regardless of where they live and regardless of where an event is held, because caching servers would cache long-lived objects irrespective of home zone.

In contrast, geographically regionalized zone servers only help those currently at home in their own zone, or in other zones sited locally. This helps not at all when they go to events in other parts of the SL world, and it actually reduces performance for everyone else.

Furthermore, it introduces unfairness. Would you want to play a competitive game hosted in a Japanese zone server, in the knowledge that the locals have an inbuilt speed advantage over you?

The same applies to US-located servers right now of course, but regionalized caches can overcome the disparity to some extent, whereas regionalized zone servers cannot.

Tinker LaFollette

I could see private sims being regionally distributed: when someone in NYC buys a private island, you allocate a server in your NYC server farm.

Other than that, I'd tread lightly when dealing with real-world regionalism in SL. Say you did set up an NYC server farm; how would you "zone" it to create a New York bias? How would you nudge East Coasters and West Coasters into using their respective server farms? And more to the point, why bother?

If you want to accommodate other nationalities, other languages, other cultures, do it by internationalizing and localizing the software. Let the users culturally segregate, if they choose to do so; but don't force it or even try to influence it, other than by making flexible tools available.

Personally, I like being on in the wee hours to find myself chatting with a Japanese student or an American expat living in China. It transports one, metaphorically at least, to the other side of the world. The "placelessness" of SL is a virtue that enables that, and I'd be sad to see it diminish.

Oz Spade

SL itself is kinda like a country created itself from many other countries and cultures from them. A good example of this is Yamato and Venice, both are based on different cultures, yet exist much closer than they would in any real world location otherwise.

To see more things like this, creations based on ideas from other cultures, would be very interisting to me. I myself am already interisted in alot of foriegn culture, so to see this braught to Second Life would be lovely.

I did see the video with Bel Muse. Its always interisting to see how close our mental images of what someone may be like in real world, based from their avatars appearence, can be, or how far off it can be, or what different aspects of who they appear to be in real life are shown through their avatars. Bel did a great job in that interview also btw.

And from where would I like to see more international creations? Hmm... I've always been fascinated by Asian cultures. Also some lesser heard of countries that have unique cultures, it would be interisting to see how they respond to and what they would create in Second Life.

Very good and interisting points Morgaine and Tinker. I have nothing that would be of value to add that hasn't already been said.

We see alot of geographical server access biased in some of the internet today, with many games having servers in different parts of the world, specificly because of bandwith speed issues.

So to figure out how Second Life will overcome such things I imagine is a hard problem, SL being more like a game than like the internet which is mostly text based and when anything greater than that comes into question we see the same issues again.

Marcus Samiam


I recently joined SL, and I'm an international user. I'm still getting used to the richness of SL, and I found your post interesting and far reaching. I have a couple of comments (not on the technical side, I think Morgaine Dinova has already pointed out some of the issues there):

1-One of the fundamental features of SL is, IMO, that by creating a world that is not our world (and it doesn't have to be) it may help break some of the barriers that we have in RL to meet and interact people from other corners of the RL world. And I don't mean only the need to travel there, we can already do without that with regular chat on the Net (or pick your favorite IM client and add a webcam). I'm talking about the rich set of nonverbal contextual cues that SL permit (gestures, animation, full AV customization, etc.) and the infinite creative possibilites that SL offers and that have no counterpart in the physical world.

I think that localized -even seggregated- communities within SL that mirror the geographical placement of their residents would go in the opposite direction. I agree that it may serve some purpose in some specific cases, but I truly hope this is not the primary evolutionary line of SL.

2-I think you're being a bit idealistic when you say "... but we inevitably then are drawn into confronting how deeply we are the same", because maybe some of the core values and events that really make us humans the same across nations and borders are (deliberately) excluded from SL, namely, experiencing the birth of a child, or the death of one's father, or tasting for the first time a delightful dish previously unknown to us (I'm getting a bit poetic now :). I agree that we can still share a lot of each of us, through stories, discussions, games, (virtual) love/sex, etc. and that SL might be an ideal vehicle for at least getting together people from different nations and cultures -btw, until a realistic automatic text-to-speech app works well that would create an interesting market for translators in SL, much like in the UN!-, but would that require that we bring our specifics (race, education, personal beliefs about nationalism and culture differences, etc.) into SL? would this enrich SL or would it turn it just into a virtual arena for discussion about RL issues?. Would it be possible on the contrary to bring in just "world citizens" in the virtual world of SL?
Some of the recent research on interactions within virtual worlds seems to point in the direction that social conventions of physical proxemics are reproduced within them; if you add to that the fundamental differences and barriers that language creates on us humans (even in our perceptions of reality), then we all are already somewhat constrained in trying to fully confront the fact of how deeply we are the same. Why add more barriers to that, instead of exploiting SL as a vehicle for constructing identities that define us without resorting to the properties that we use in RL for that, such as nationality, religion, social status, and that more than often are the source of what separates us?

Finally, following the "Imagine" theme, I can't resist to take one giant step further and imagine that the future history of SL could well resemble the sheer magnitude of Tlön, an imaginary world described in "Tlön, Uqbar, Orbis Tertius", a short story written by Jorge Luis Borges in 1940 and first published as part of a 1941 collection of stories called "El jardín de senderos que se bifurcan" ("The Garden of Forking Paths").

I've written down already a few pointers on this that I think you might find interesting (Disclaimer: Philip, I'm not saying that you're the alter ego of Ezra Buckley!!)



Morgaine Dinova

Very nicely put, Marcus.

Please let's use the expanding set of objects and groups and interests in Second Life to create our own marks of Second Life individuality. We really do not need the divisive First Life differences of race, colour, nationality, religion or anything like that. We've suffered enough from it in 1L.

There is a fine line between importing a few pretty elements of 1L culture and importing/supporting 1L nationalism or fundamentalism. Be very very careful, because there is no going back once a mistake is made.

Alexin Bismark

Responding to Morgaine...

The great thing about SL is that, generally speaking and operating within the TOS, people are free to make those choices themselves (i.e. what they carry over from 1L) and not be subject to personal preferences or prejudices of others. It is my belief that precisely this is what can help people learn to understand some those 1L differences in a "detached" and "safe" reality like SL. And this can perhaps help reduce (if even by a small amount, and on the level of individuals) some of that 1L suffering. I'm thinking maybe thats what Marcus may have been getting at.

People being people, I don't for a second believe that you're going to be able strip out things like people's racial, cultural, national or religious identity at the door. Nor should you try. These things and others are what make people the colorful human individuals they are. You don't learn to be tolerant of diversity by making everyone the same. That always leads me to think of the short story "Harrison Bergeron" by Kurt Vonnegut. Sure, people are nice and safe and never offended or "made to feel bad" in Harrison's world...but gods...at what a price.

I like colorful people, with whom I will disagree and who will occasionally piss me off over the "nicer" alternative. And thank goodness the Linden's have left us all to our own devices in that regard. And I think it would be cool as hell to see a wiccan temple on a parcel right next to a christian cathedral, across the street from a mosque and a shinto shrine.

Strife Onizuka

Alexin, to get those all in one sim there would have to be some incentive, (or it be an island sim). A religion sim would be very interesting (dispite what is common belief; sex, gambling and shopping don't count).

The localization idea is interesting. But that brings to light the logistical issues of maintenance of the clusters and the legal implications of having servers in other countries.

Philip Linden

I over emphasized the idea of separate servers - this isn't really the important point, and I agree that a powerful thing about SL is that everyone lives everywhere. The idea of putting physical servers in different parts of the world would be technical, not social (read more below).

So the first order of business (I think) is simply making it more easy for people outside the US to access, understand, and be able to pay for SL.

Beyond that... very soon we will get the frame rates on both viewer and simulator up to the point where SL feels like an FPS engine in terms of speed. Once this happens, we will all notice that the ping delay in the network will be the most objectionable source of lag. If you are in Russia and you click on a piano key that is on a server in San Francisco, it will be 200 milliseconds or so until you can see it move and hear it play. This is due to the delay in packets moving from Russia to US and back. Some of this delay is in networking equipment, but there is a lower limit set by the speed of light... about 120 milliseconds worst case (halfway around the world). So this means that there will be appreciable lag for users connecting to servers that are geographically distant. Putting servers in various parts of the world would mean that everyone would at least have SOME areas in SL where they got lag-free speeds.

Alexin Bismark


Sounds like a first generation step to something like this would be a European Grid (or South American Grid or Asian Grid), equivalent to the current "North American" Grid, with a portal that allows AV transit between grid. Much like the early days of intercontinental travel, when people crossed the seas in wooden ships, it started small and grew. While "travelling abroad" was sure to bring the excitement of new adventures and new people, it wasn't quite as easy or convenient as being at home, so those that chose to travel expected the hardships along the way. The mass of people staying in the comfort of their homeland. Content to see and buy the amazing products found in far off lands brought home by their adventurous neighbors, and listen to their strange tales of a place where people ran around naked and (cover your ears Beatrice) display their genitals on the tops of their heads! Amazing! Call me Marco Polo, where do I signup?

Anshe Chung

An important aspect of international distribution of servers would be that they could fall under local jurisdications. This would make it more like the WWW where no single government can control the content of the whole Web.

To make SL truly international localization is needed. Very basic things such as support for local keyboard settings or multi-language documentation and customer support. As things are now, SL may have 25% members who are not residents of the United States. But most of them are still part of or closely related to the anglosaxon culture. This makes the SL population geographically
spread, but not necessarily culturaly diverse.

Tiger Crossing

I've been around avatars that were speaking a different language, one that I didn't understand. While tolerance for other cultures' differences is a must (ToS alone, if not common curiously demands it), the will always be friction when cultures collide. I've found that even the most tolerant of individuals can be upset when surrounded by things, words, and ideas that they don't understand and can't easily learn. A conversation between two avatars in the welcome area in a non-English language is a good example. Those present that don't speak that language will feel that tingle of ignorance and resent the fact. Most won't let it bother them, but it is probably there, nonetheless.

If Second Life could include a text language translator in-line and live on the servers, translating in real-time based on individuals' chosen language of preference, it would go a long way to making all languages and cultures accepted in a single virtual world. I'm sure native speakers will know when the text they are reading is coming out of a translator, but there ARE good translators around, and it's much better than having nothing at all.

heheh I could even see someone making parrot that contains a script that listens expecting one language, translates it to another, and spits it back out... Learning a new language, the vocabulary in particular, but experimentation. With such tools available, and SCRIPTABLE, I can see a large portion of an international Second Life population becoming at least partial polyglots. :)

(As for remote servers, they can still be part of the main grid. Just connect them with a couple of Void sims. I imagine that social interactions will happen anywhere, but there would be Jetball stadiums for each geo-region...)

Morgaine Dinova

I like your idea of translator parrots, Tiger. And I have no doubt that someone will create Babel fish too. :-)

This also brings to mind something that people have been requesting like forever, and that is vicinity chat and IM logging while online, so that (among other things) local text-to-speech clients can tail the logs and speak the text out loud.

Allegedly there is some dogma in LL about not recording text. In reality there are no extra issues of substance in this area since chat and IM text can be scraped off the screen and pasted into logfiles anyway, so any such alleged dogma is misguided.

Let's have logging. I want to run Festival on the output and hear people, and maybe feed it to my Babel fish before that. :-)

eggy lippmann

I would bloody murder for a european server. God, a 250ms ping is pretty much the standard for me. The building tools feel kind of jumpy sometimes, snapping back and forth when I move something. Why can't you trust my input to be the correct and desired position for an object, instead of refreshing my client with the erroneous server position? It's my object, on my land. I dont really care if people see it on a different position from mine, I usually dont even have people around to see me building.
I wish we could separate development and publishing, instead of streaming everything back and forth in real time all the time. Maybe one day it will be possible to build on a local sim and then upload the object to the grid. I mean, its how the web works, and it works fine. It's how everything works in the real world.
I would gladly settle for a european server, though! :D

Carnildo Greenacre

How about starting out by moving some of the sandboxes? Move one to southeast Asia, and one to Europe. The sandboxes are hardly a critical part of the system, so if something goes wrong when setting them up, or if connectivity problems cause one to drop off the grid for a while, it doesn't really matter. Name them to indicate where they are, or give a message of "This sandbox is located in Europe. If editing feels a little sluggish, try a different sandbox".

The benefits are that you'll get some real-world experience with setting up a distributed grid without impacting playing experience significantly, and that overseas users will have a low-ping place to build.

Gwyneth Llewelyn

I hearby state that I will gladly manage voluntarily a European server with at least 2 Mbps upstreaming bandwidth hosted somewhere in France :) That will be probably enough for 20 AVs at the same time. Just send a tarball with the source code to [email protected] :) and I'll have it up and running next weekend :)

Jokes beside, the people who have met me online or take the time to read my blog, knows how I feel about SL being the building blocks for the Metaverse, and certainly the first step is trying to get sims running on other parts of the world...

Easy Wheeling

World Peace through Massivly Multi-player On-Line Gaming has been my hope since the early days of Ultima Online.

You are on the right track, and proceeding faster than I hoped. Yes, a translation engine is key. Target markets and supported languages will likely need to be market driven, however, consider the possibilities that could be realized by also including Chinese, Arabic, and languages appropriate to other cultures struggling to find their fit in a modern world.

By the way, what you have enabled and are enabling for the human race with this expandable user owned and created universe brings extraordinary posibilities on so many levels. Stay true to it, and history will forever record where it began.

Now, when can we get SL hosted on something like IBM's planet-wide distributed super computer arrays?

-Easy Wheeling, Olopue.

Prokofy Neva

I'm going to introduce a note of skepticism here. I think that the breathlessness and enthusiasm that comes with crossing international boundaries are possible among tekkies and game geeks because they are encountering other people at the edge of those foreign cultures like themselves, with something in common, namely programmer's language, and the arcana of online games.I think if you really learned foreign languages and lived in foreign countries IRL, you'd have a more realistic few of both the opportunities, costs, and threats of intercultural communication. I just have to say that you probably have never deeply befriended or fallen in love with foreigners in RL or sim life, and so you have no idea of the vasty deeps of cultural differences and the accelerated and intense potential for conflict in an online setting. Just study the conflicts that develop even among more or less culturally akin people in the very specialized population of SL to get a faint glimmer. Easy proximity does not necessarily mean understanding and wisdom. In fact, instead of an integrated and comprehending Global Village as Marshall McLuhan described it 30 years ago, we have seen that the dissemination of images and information quickly throughout the world is just as liable to lead to people who get the idea to come and crash planes into the World Trade Center as it does people who want to have "world peace" -- whatever THAT is. In many non-Western countries, the numbers of Internet users and people with the leisure to play online games are in the single digit percentages. The idea that people in say, Darfur, Sudan or Arkhangelsk, Russia who have never been near a computer are going to be "better" at investing their consciousness in avatars seems patronizing -- what, they have animistic or paganistic primitive beliefs in things like tree spirits, so it makes it culturally easy for them? You have no idea what it is like to spend quality time with hordes of people, mainly uneducated and uncultured, who do not at all share your liberal lefty American values. They will eat you alive for breakfast, lunch, and dinner. They are more likely to tend to fascism and fundamentalism than they are to Esperanto-type utopian internationalism. A few like-minded individuals might build bridges to each others' societies in games...but they will remain isolated. In many countries of the world, getting shot at, losing all your money, and losing you clothes and objects, are not entertaining things in a game or just a little bugginess, but their real lives. You haven't even begun to imagine how the revolutionary device you are working on now will fuel the Clash of Civilizations just as much as it will help to avoid it.

Noel Wade / Tread Whiplash

Prokofy -

I agree with you that an online arena can never replace the experience of going to another country and getting to know its people first-hand. I've done that, and its important for many of the reasons you have stated.

HOWEVER, here in the USA, we're a big country that's surrounded by oceans. Anything involving a country other than Canada or Mexico is "over the horizon"; and is not paid attention to, by the vast majority of the public. I agree with Philip that cultures in SL (and the social context that it provides) can be an aid in getting people of different backgrounds to understand each other better and be more "Globally Conscious" (to use a buzzword).

Its not a substitute for first-hand experience. Its not going to solve the world's problems. And its probably 10 years before this sort of technology is as common-place as webpages are now (and its been a decade since the web was introduced)... But look at how Blogs and professional news-service websites have changed the way we can track and recieve international news. Doing so "face to face" in a 3d environment will be as compelling, if not more so - since it will again be a step forward in interactivity and active participation in world events.

The key will be lowering the financial, physical, and social "barriers to entry" over the next few years. But I believe it can be done (if taken on at a realistic and incremental pace); and that there will be benefits on both the commercial and the social/cultural side of things - which is why I applied to Linden Lab today. :-)

Take care,

--Noel Wade
("Tread Whiplash")

Fritz Rosencrans

Always a complex theme, interaction with other cultures! The very refined layer of political correctness, in the widest sense, that has been achieved within smaller and larger subcultures, is completely absent when an external culture is involved. Issues that have become Holy Cows or Totally Taboo in one subculture, may be fair game or simply normal conversational time-fillers in another. The fact that there are languages dividing many cultures protects these no-go zones to a certain extent. However, this is being weakened by the common interface on the Web, the English language.

The result is BOUND to be friction. However, I see this as a very valuable way of opening the no-go areas that everyone has in their minds, more or less. And this opening is bound to be painful.

The advantage I see, besides the lag issue, in having servers distributed around the planet is the necessity for an abstraction layer, both for load sharing and the proxy function - which has one BIG advantage: we may get an SL that can support the various international keyboards, even if the prime and central language of SL remains English. And I DO believe the base language of SL should remain English. Translations a la Babel Fish are fun, and amusing, until you need an EXACT translation.

If a common medium of interchange is not maintained, and I mean the English Language, there is a great danger of Ghetto Building, or Island States. And this is for me a horror - the huge strength of SL has always been the openness of the whole world.

The idea of Germany for example being handled by a purely German Server is for me also not good. This encourages a national splitting for performance reasons, and acts as an additional barrier to integration. Then you get national laws in various countries limiting or determining the information flows and contents on these national servers, and the Ghetto is complete. The idea of Proxies and a more transparent load levelling system, where who knows which sim is on which server, and could care less, is a more scalable and socially far more powerful concept.

To this end, having the sims divided into "global" (static) and "local" (dynamic) parts would enable the global parts to be stably cached, and the interchange to the "actual" sim would then only be on the dymanic part... usually a far lower data volume.

Well thats it for 2 cents... I am deliberately speaking off the top of my hat here, as I do not know if the SL logical structure will support such a system. But I am all for virtualization, dynamic load levelling and so on and Very Much against localisation of content servers, unless it proves necessary for risk management.

Reality Bites

Please bring SL to Japan! I am a North American (both Canadian and American citizenship) living in Tokyo, Japan. The other night I was playing and came across a fellow Japanese Tokyo-ite. His (her?) English was superb, and I envied his ability to play SL in an English rich environment. I wanted to do the same in Japanese I'm sure I could join a different Japanese online community, but I want my Second Life!

There is a huge market in teaching English in Japan (think McDonald's). Games like this on an computer with a decent English/Japanese dictionary is like free, fun, interactive, productive?, profitable? study! The potential for real conversational English, in all imaginable contexts, is just incredible. Need anyone to promote SL here? I would love to do it!

A few tweaks and SL could become a "language teaching aid". Imagine groups of Japanese avatars going around SL doing screen saves, rather than taking photos. Japanese are generally shy about speaking English, and the phenomena/stereotype of Japanese in large groups taking photos is based on the cultural value of safety. The Japanese tend to see themselves as "innocents abroad" when traveling.
With a largely wired populace, and with all the incentives to being part of the online world. I assure you, that this is going to be one of the most dominate online markets in the world.

Yoroshiku onegaishimasu!

daz Groshomme

hmmm I'm not sure that a particular culture is going to care about recreating their own reality for us to learn anything from, BUT it sure as hell is interesting to see other culture's imaginations at work. I haven't found any international communities yet but the first person I 'talked' to was from Japan, it was a cool experience!!

Anonymous Coward

I wonder if other cultures will be repulsed by the amount of porn and casinos?

The comments to this entry are closed.