Server Backstage of MMO Games

There are some sweet topics in high-tech which easily catching the attention of the general public. Just try to do a talk on some trendy AJAX framework and most people who are not in the web development won't be interested. Now try the same with SQL optimizations and you'll lose everybody who is not a back-end developer. Maybe some topic for starters? A lot of experienced people won't be interested. Tricky details of GC optimization for JVM? And only experts would come up. Seems like every talk has it's target audience.

But I've found that some topics will appeal to a wide audience no matter what. One of these magic topics is MMO game development. Whenever I mention my involvement in MMO world I always get a lot of interest. Even from people who have never worked in the game development. The topic is magnetic and everybody wants to get some insights on how MMO games are ticking.

Actually, you don't have to be deeply familiar with technology to understand the number of challenges MMO game developers are facing. There are thousands of players, huge traffic, latency, synchronization, load balancing and a lot of other issues – both technical and social. MMO games are enormous projects requiring a lot of people to be involved. And also money... A lot of money...

Sure thing that would be fascinating to everybody! Even for people who is not that much into technology. I had artists and game designers asking me how the dark server side is functioning.

This September I've been able to do a talk on server side of MMO games. A cool company Ciklum held GameDev Saturday in their Dnepropetrovsk office. It attracted a lot of people involved in game development – actually the best target audience for a talk on MMOG. Since not only developers were attending, I tried not to dive into technical details and keep it catchy for everyone.

I've covered basically the whole life cycle of a big MMO project from server-side perspective. I used my experience with Cities XL - it was one of the best projects I've participated in and I tried to reflect all the positive vibes on the audience.

Now the video of my talk is available here (the talk is in Russian) along with slides. There is also a photo album of the event.

Kudos to Tatyana Prudnikova and other Ciklum guys and girls for the effort in organizing the event. It was remarkable experience in the remarkable city on the Dnepr river :)

 

Overstructuring

Today is 0xFF day of the year, meaning a holiday for anybody who can read in HEX. Therefore I'm going to present a little essay on directory structures and their overcomplication.

There is something magical about complicated structures. They seem to have lives on their own, attracting people attention like magnets. Their existence is predefined by the problems they are trying to solve. Some problems are really that complex and can't be simplified. But often engineers keep building complicated structures where the simple ones will do.

I will take Maven as an example, since I mentioned it in my latest posts. But you can find many other examples throughout the industry and beyond.

Maven Standard Directory Layout is stupid. It is an example of unnecessary complication. It is too deep with too many subcategories. I agree that the structure is smart and it fits well... For a big project. But the fact is that most of projects are not that big at all. Some of them are small, some are bigger, but none of these would really require the layout offered by Maven. What you end up with is a bunch of directories containing only a single subdirectory inside. And if this is the case, there was no point in the subcategory in the first place. The layout is overstructured.

Yes, I know that you can redefine the layout if needed. But the fact that this is the default layout and Maven "strongly recommends" using it for your projects makes it de facto standard, as most developers prefer to stick to it.

Just look over several Maven projects to find out that most of them have overstructured directory layouts. Actually, I can assume that 95% of them do. And only 3% will be able to fill that structure for real. Maybe somebody is wondering where is the missing 2%? These are the projects so complicated, that the directory layout offered by Maven is to primitive.

So basically Maven Standard Directory Layout fits only the small selection of real projects. Bad work in selecting your target audience, Maven guys! No wonder it took so much time for Maven to get into the mainstream since it's introduction in 2001.

Their layout is a classical design-by-committee example. Who said it is the right structure? It is claimed that number of industry experts are participated in defining that layout. I have no doubts on that account. Probably it was like:

"...we need source files and tests."

"Yep. And also place to store configuration files..."

"What about filters?"

"...and don't forget webapps!...".

Everybody tried to include their own vision of the structure. And the result of a compromise is an overcomplicated layout - with one folder for each expert!

Now, somebody could argue that common layout is necessary for developers to feel familiar with any Maven project, it requires less configuration and it also makes tool integration easier. All these things are true, but that is not an excuse for a complicated layout.

Most projects require only folders for source files, tests, configs and optionally for scripts and resources. It is logical to provide a simple layout which fits the majority of projects(95%) and at the same time give means to redefine it for complicated ones. It is possible to provide several layout profiles for different kind of projects. But the simplest one must be the default choice. Sure it would complicate tool development, but we are not in the business of making tool developers life easier.

Right now I'm promoting Gradle as the best integration tool for Java projects. No XML and clarity of DSL makes it a joyride. To my disappointment Gradle also inherited the Maven layout instead of rethinking it. Committee has prevailed over common sense and even Gradle couldn't resist the magic of overstructuring. C'est dommage :(

KISS! And happy 0xFF day! We're still controlling the power of machines... Are they able to control us?

XML Overdose

While giving a talk at IT-Jam 2011 in Odessa I mentioned that XML is poisoning our lives (Russian "...портит нам жизнь" to be accurate). And apparently some people got the impression that I'm in favor of getting back to monochrome ASCII terminals or something...

Well, allow me to clarify my perspective on that topic. I'm using XML extensively in my day-to-day life. And I think it is a clever syntax to present data. Maybe a little bit overloaded with meta information, but clever anyway. It has it's own weaknesses, but in general it is a good data format to use. It is a standard, its backed by a lot of tools and it is ubiquitous throughout the computing world - supported by most of modern languages, frameworks and libraries.

My attitude toward XML was changing all the time I was using it. First it was confusion from that mystic way to organize data - it was hard to understand the concept of semantic mark-up since years of HTML formatting. But when the idea became dominant I started to appreciate the power of XML and tried to use it everywhere I can - transfer protocols, data formats, configurations... It soon became apparent that I'm actually writing real programming languages in XML. And a lot of people, including framework developers, doing that too. So we ended up in the world where we are doing more declarative coding in XML than coding in imperative programming languages... And some ideas are taking much more space to express in XML, so it does not make things concise.

Once I realized that, I switched from blind injection of XML into anything I could to a more pragmatic approach. I think we are overusing XML a lot these days. And it fits well for some purposes, but for others makes things more complex. By trying to fit XML where it doesn't belong we are loosing a sense of simple solutions and getting the XML overdose.

And that XML overdose playing tricks with our brains - just look at the Maven dependency declaration:

<dependencies>

    <dependency>

        <groupid>junit</groupid>

        <artifactid>junit</artifactid>

        <version>4.0</version>

        <type>jar</type>

        <optional>true</optional>

    </dependency>

</dependencies>

To set 5 values (ids, version, type and optional) we have to specify 7 strings. And in a real project we would have 10s if not 100s of dependencies each requesting 5-8 strings to define. That's what I call the XML overdose!

In this case Maven is blindly following best recommendations you can find at W3C Schools. They suggest to use elements instead of attributes, since elements are flexible, extendible, can have multiple values etc... When do they propose to use attributes? Well, when you have some identification values like NAME or ID.

So, you can look at it this way - groupid, artifactid and version are like identifiers for a dependency. So you can put them as attributes. Developers of Apache Ivy have done just that. And the same declaration in Ivy would take only one XML string:

<dependencies>

    <dependency org="junit" name="junit" rev="4.0" optional="true"/></dependencies>

We can get rid of XML at all. And Gradle by doing that makes the declaration even more concise:

dependencies {

    testCompile 'junit:junit:4.0'

}

Now that is a big needle to stick into anyone affected by the XML overdose!

Just ask yourself - why do I need bloated syntax to define my project dependencies? Only because it is a STANDARD and it is a way most developers doing it?

I'd better keep my own perspective on this matter since I'm tired of tag-bloat in my pom.xml files.

If you are designing an XML format humans are going to work with, try to keep it as simple as possible. That is the only ground rule here. Avoid unnecessary structures and nesting, try to minimize mandatory values by using defaults. And most of all - DO NOT use XML namespaces - they are making tags ugly and unreadable. If it is possible, consider using some alternative format instead of XML.

In summary, with some effort on our part, it is possible to make XML data confortable to work with. Just do not overdose it!

P.S. Don't get the impression I have something against monochrome terminals. I'm a fan of retro-computing. Maybe I missed the glory days of PDP-11 era, but I had my time programming ZX Spectrum in early 90s. And it was a great time with a really neat machine!

IT Jam 2011 Report

Last weekend I attended the IT Jam 2011 conference in Odessa. The experience was so intense and vivid, that I'm still recovering from the trip and only recently have managed to sort out all correspondence gathered while I'm gone.

Odessa is a great place for such a conference and it attracted 1000+ IT professionals. Before the opening, I've managed to get my caffeine dope from a great café in the downtown.

Odessa is able to impress. It welcomed us with great sunny weather, distinctive architecture, impressive industrial skyline of a port, good restaurants and dynamic night life.

The conference itself was hosted in the marine passenger terminal - probably one of the few places in Odessa capable to hold an event of that magnitude.

 
The talks were divided in 6 streams - Main stage, Java, .Net, Web, Mobile and QA.

I was giving a talk in the Java stream called "Gradle - next generation of build tools". It was a lot of fun - listeners were very positive, easygoing and responsive. And according to tweets the talk was accepted quite well.

The only thing I regret is that I haven't positioned the talk properly. And some people got the impression that I'm trying to persuade them to migrate on Gradle right away (which I'm not). I just wanted to increase product awareness, so Java developers would mark on their radar another convenient build integration tool. And it makes a good sense, since according to my initial quiz nobody uses Gradle.

Along the way I've mentioned Polyglot Maven as a way to handle pom.xml complexity. Apache Ivy also came into view - I was really surprised, that nobody actually knows about it.

My presentation is available at SlideShare (in Russian). I actually made a really fun mistake in the code, which hasn't passed unnoticed. While making a "hello world" program for Gradle I first made it say "hello world!" but than decided to change it to "hello Gradle". However, I've forgotten to change the original source, only the output has been changed. So I ended up with println "hello world" seems like printing "hello Gradle". My appologies :)

The conference was very dynamic with a lot of people going back and forth between the talks. I've met a lot of interesting people and had the opportunity to chat with folks I haven't seen for a while.

There was some problems with food availability during the lunch. I, however, skip some talks to visit the summer lounge in a good Odessa restaurant.

So I was really surprised to see all the whining in twitter that all food is gone. No, it is not! Just get outside and find a good restaurant to eat - your are in Odessa after all!

But somebody expected on free conference to get free food to boot. So naive and immature on their part. As for the conference organizers, they probably should have cleared more time for lunch. I don't have any other complains with organization at all - the event was great and all small inconveniences was inevitable for such a scale. They were caused by the complexity of the event itself and not by the lack of organizational effort (people complained a lot about sound isolation in improvised stages - well, just try to find a better place to host 2000 people!).

IT Jam was closed by a spectacular rock show featuring several rock/punk/metal bands consisting mostly of IT-people. The sound was harsh and loud - exactly what we needed after the whole day of tech talks.

After last chords everybody could enjoy Odessa nightlife.

Italian tall ship "Amerigo Vespucci" was in Odessa harbour at the time. Truly sailing work of art attracting both locals and tourists.

You can find photo report from my trip here.

In summary, I want to thank Ciklum for the effort in organizing the conference. The jam was a fascinating event. It was worth the time and energy to visit. And I'm looking forward for the next year jam. Let it be bigger, brighter and louder!

JEEConf slides and videos

In May I had the pleasure to be a speaker at JEEConf. That was an extraordinary and exclusive event. We've never had anything of that scale in Kiev before. Big thanks to XP Injection for organizing the conference, to all speakers for great talks and to hundreds of Java developers who showed up there. The conference was a constant flow of information and intensive chatting with fellow developers during the breaks. And I was enjoying every minute of it.

Now we have the videos and the slides for all the talks available on-line. So, I finally can check out the talks I've missed and enjoy the talks I've attended one more time.

My speech on Domain Specific Languages is here (it is in Russian).

You can also check out the great presentation of Apache Camel by Claus Ibsen, the deep tech details of HotSpot garbage collector tuning by Vladimir Ivanov from Oracle (in Russian), awesome rebel talk on graph databases and Neo4j by Andrés Taylor and all other great talks (mostly in Russian).

I'm looking forward for more events of that kind here in Kiev. Our developers community is really need it.

Speech at JEEConf

Tomorrow I'm going to give a speech on DSLs at JEEConf.

It is going to be about pizza, terminators and voodoo magic. But basically on how we can improve our understanding of domain and make the architecture more flexible along the way. And how voodoo magic (compilers and interpreters) makes it all possible.

This is another key event for me in the following weeks, so I'm looking forward to it :)

Landscape is Changing

The recent turn of events are changing the landscape drastically. For 11+ years I've been in this industry it's never been that dynamic. At least for me...

New technologies are creeping into the scenery - dynamic and functional languages are going mainstream. DSLs are getting spotted on the radar too. And some of that tech is promising to simplify the development a little.

I know - there were all kind of silver bullets in the past - OOP, 4GL, RAD etc... And although nothing has revolutionized the business, there have been countless little improvements to make the life easier. The problem, of course, that with these little improvements new tech sometimes brings a lot of over-complication. Entry barrier in the industry is high as never and existing tools don't help much. So looking at that I'm often thinking whether it worth the trouble.

My last projects and recent coaching experience forced me to rethink how I evaluate, learn and apply technology. And given the nature of evangelical and design roles I've played in the recent projects, I've accumulated some useful tips in the process.

All my recent experience plus changes in the tech landscape is the reason why I'm refocusing on the new horizons. And this blog is the result of that new focus.

Now I see the landscape more clearly than ever before. And I'm going to cover that wild trip here :)