Dune II allowed as college project!

After much deliberation at my college, I managed to get it accepted as a project that will provide me with some grades that I need. There are 336 hours scheduled for Dune(Which comes down to 42 full days, or 1.5 month of work fulltime).

This should give me plenty of time to actually get anything done on the project. However, after reviewing the old code and an attempt to get it to compile again under Visual Studio 2008 on Windows 7, I have decided that certain parts of the game are utterly useless. Managed DirectX support appears to be dropped by Microsoft, and is replaced with XNA.

Because of this, and the insane difficulty I had with integrating a network component into the game, I've decided that I'm going to partially re-write Dune. I will completely replace the rendering layer with XNA, and I will start bottom up this time. Instead of working on the game logic at first, I will start out with the networking engine, and build a game on top of that. This allows me to focus on writing a solid network component first, and then slowly integrate parts from the game into the simulation.

However, since I spoke of a partial re-write, you might wonder what I will re-use. Well, I will re-use all the logic components I've written till so far. I will still use many of the components from the original engine(Such as the manager for gamestates, tasks) and try to port as many things over to XNA. One of the first things on the list is the Window Manager because I'm going to need it on short term.

I started testing out some code on networking and NAT punchthrough last monday using the latest Lidgren Network Library, and I must say, it's great! I wrote a little test application that creates a P2P network with 1 host and 2 clients, and I was able to generate a solid, working p2p network. They all saw each other, and it could connect over a home router and a corporate firewall(Microsoft ISA). The codebase I know have, is derived from that, and has a working lobby that recognizes new players, sends game settings around and at the moment, I'm working on setting up an actual simulation, that will start out by running a few rounds, and the host sending 1 to 2 orders around to all clients, have the clients execute these and then have them send the results back.

I'll report back later. I hope to have some screenshots by then too.

Synchronisation bug in the server...

Well well, what a surprise! More updates on Dune :).

I've coded a bit on Dune in the past few days. Not an awful lot, because I had other things aswell, but I managed to code up a few bits on the train while traveling to/from college.

Units are moving again

Yes, first thing I did was re-enable the controls to move units and scroll around the map. Sure, they don't actually send stuff over the network, but units are moving, and I found a few bugs in there related to the MCV(ie, MCV can attack, but isn't supposed to). I intend to fix this first, before actually buffering the calls to order units around.

I also need to build the turn-system for the packets. I guess I should buffer up the commands first, and then actually start working on the system that collects the packets from other sources and hands down the orders to the correct player.

DebugConsole is now a real UI object

I updated the DebugConsole, because it didn't quite behave like I wanted it to behave. It was the first 'UI' control I built, and wasn't based on the later written UI system. In other words, it was a dirty hack that rendered itself to a texture and constantly drew the texture when needed. Obviously, that was faster than the current approach, but made it more difficult to make the console support features like real UI objects, such as resizes and adding of sub-controls.

I think the next iteration of the console replaces the current code with a Textfield / Textbox in which I draw the text, and a 2nd textfield / textbox to type in. That would make it much easier to add in support for scrollbars, and other various snazzy features. But, that doesn't have an awful lot of priority at the moment.

Synchronisation bug!

And of course the bug I mentioned in the title. I have a really weird bug in the Server code related to loading. It's a few days old, and I've been trying several times to patch this up, but it didn't quite work.

The problem is as follows: When I enter the NetworkPlayState, this will create a new state called NetworkLoadingState which is responsible for loading all assets and synchronising all the clients with each other so that each game starts around the same time. However, when the server is behind on loading than the client(On my machine that is. The game having the focus will load slightly faster than the background window), it will be done loading by the time all the clients are done loading too. It will see that everyone is done, and trigger the OnStartGame event, which in that case, will report that the server hasn't completed loading yet. This will mean the client will push a new NetworkLoadingState onto the stack and voila, the game will start throwing exceptions.

I have no idea yet on how I'm going to solve this. I'm considering the just ditch the current asynch approach, get rid of the events of the events and start using blocking functions instead, to make sure things will run in proper order. Alternatively, I might just build in a 1 second delay on the server side to prevent this from happening, but that would be a dirty hack I rather avoid. First step is to figure out where it actually goes wrong, and then find a way to solve the issue neatly, without fudging up my code too much.

Unexpected progress!

Some people probably gave up on me(or Dune or both), but don't worry, I haven't given up yet. It's just that I've been way too busy with other things in my life(Which got twisted upside. But I didn't move in with my auntie and uncle in Bel Air).

Last friday, I had my laptop with me and I was like, what if I open up Dune and start coding for a bit again? And so it happened. I coded, and debugged the code I had written, and debugged more, and yelled at my laptop. I really had to look back at older code to see how far I was, and how I solved certain issues. And to be honest, I solved quite a lot of things in a clever way.

Last weekend, I managed to finish up the lobby / loading / synchronizing part. It now successfully manages to synchronize all the data between the lobby/clients, and then starts the game, including units. Each player gets an auto-assigned starting location when they connect(At the moment, since I haven't added in start points to the maps yet). After the game is done loading, it shows everything like a regular single-player game, except the controls are currently disabled.

Lobby-server awaiting another player to join. Player numbers are currently fixed.

Minor bug with the sidebar during loading. Still need to fix that, but it's not too important at the moment.

And we're live! This shows up on both sides.

Next step is to re-enable the player controls. But instead of immediatly executing all the actions, I need to buffer them, send them to the server and have the server distribute this to all clients again. I already isolated the calls which I need to buffer, so I only need to build the mechanism to actually buffer them, broadcast them around and execute them when needed. Still need to figure out how I'm going to do that.

Happy new year! (And progress updates)

Happy new year! May 2008 be a good year!

As in updates: I've managed to complete the loading of all assets and the game actually ends up being started. So, the next step is to place some units on the map and start building the turn system.

I'm not entirely sure on how I'm going to implement the turn-system yet. I intended to send a StartTurn to all clients, which then starts each turn. But, as lag increases, the game would hang for a short period of time with this system(Perhaps not even noticeable). So I could send them out sooner, but that would mean I could miss packets with commands in them. Not sending StartTurn packets is even worse, as the game would continue to run(And go out of synch) when the connection is temporarily congested. Not a good thing either.

I think I need to experiment with this a bit, and when I have a good working alpha test release, do some real testing with 2 or 3 people to see how it works out.

I hate coursework...

There's a small delay in finishing up all the Game handshaking and getting the initial scene up. I'm currently occupied with a load of coursework which I have to finish because I screwed over a bunch of courses last year and the year before.

Really stupid, but if I want to start my graduation by the end of the college year, I better get all these courses patched up. And because of that, I have less time to work on Dune.

But, I found out I also have to re-do Network Programming(Why I failed that course is still a complete mystery to me), so I'm going to attempt to convince the professor to allow me to submit the network code of Dune. Mainly because it's more cooler than writing lame ass chat server(Using UDP and RMI), but also because it gives him a really cool example to show during classes.

So, I need to finish 1 more assignment for data structures / algorithms(Write an AVL Tree), and then I can submit all the code. One down so to say.

I need to re-think my class structure a bit...

I was reviewing my network code in the train on my way home today, and I came to the conclusion that I should think it over a little bit before I start expanding the code.

At the moment, on both the hosting and the connecting side, I have either a Server or Client as Task. The task is placed into the TaskManager and therefor updated each frame. Both the Server and the Client implement various properties via an interface(INetworkConnection). This is the connection status, connected players, etc. This is fine.

However, I also need to receive information back FROM the Task, such as when a new client connects, or when the user interface needs to update information. For this, the INetworkConnection is insufficient, because I need various events which come either from the lobby or the game. Such as PlayerConnect, PlayerForfeit, etc. It's a really bad idea to go from lobby -> Server/Client -> LobbyState. Messages would be re-routed via the INetworkConnection, which would slow things down AND clutter the INetworkConnection interface with too much useless crap.

I intend to counter this by creating ILobby and IGame interfaces, which the LobbyServer/LobbyClient and GameServer/GameClient would implement. These interfaces could then contain the function specific things for the lobby/game(such as a joining player for the lobby, and a forfeiting player for the game).

Using this system would probably be more efficient in terms of code maintenance than cramming it all into 1 gigantic interface. It would also prevent the Server/Client task from having to add GameStates, which keeps the seperation of UI / Logic intact.

I should have some in-screen screenshots soon, altho I probably need at least till sunday to buffer up all the player interaction with the game and send these commands over the network to the server. I need to change quite some code in the PlayerInput.cs file(Which basically contains all the code related to interaction from player to game).

Background loading of assets

Before I wanted to start out with the actual networking part, I wanted a way to load the assets from disk in the background, and before the game was actually running. This to prevent slowdowns in the network simulation when a new building/unit was constructed and the asset had to be loaded from disk.

In order to do this, I wanted a nice front-end(A progressbar to display the loading progress) and a back-end: A background thread which did the loading and put each asset into the TextureManager.

I started out with the ProgressBar, as the asset pre-loader was half done. It just needed to be modified to be able to work in a thread.

The progressbar is just another Control for my game. It basically draws it's outline, and based on a Value it shows a nice (blue) bar. I initially had a problem with the outlines not showing up. But, this is related to the way how D3DSprite and D3DXLine render. I solved the issue by doing a Sprite.End() RenderLine Sprite.Begin(), but that cuts down on performance, so I need a better way. I reported it as a 'tweak' in mantis, so I don't forget about it.

Loading, please wait...

The background asset loader was also heavily modified. I initially wanted to try and run it while performing my normal gamestate functionality, but I found out it's way too slow for that. The change was that when I constructed the Preloader, it would parse the list of assets to load and store these, and then with a call to LoadNext() it would take the next asset from the list and load it. So, I shoved that into my BackgroundLoader class, and all was done. The BackgroundLoader is a thread that keeps running until all loading is done, and once it loads the next asset, it will increase a counter, so I can poll how many assets are done and then update the progress bar.

However, I was quite shocked when I tested out the background loading process. It's slow as hell! It took 94 seconds in debug build and 87 seconds in release build to load all 58 assets in just 2 colors. Image how long it'll take when you're playing with 6 friends...

Slowpoke is slow :(

So yea, background loading is in, except it's terribly slow. In fact, it's so slow, it's unusable to develop the network play with, because having to wait ~1.5 minute before being able to test something out is too much for me.

For the coming tests(The actual network gameplay), I'll be testing with a reduced preload set, just a handfull of units / buildings.

A peak into Dune's protocol

I'm currently happily coding away at the Lobby for the game. Most of the code is in now, with the exception of colors / factions / map. I'll add in which map will be loaded soon to the host as a command line switch (dune -host 2 -name Host -map mp_test.map). I also added in most of the code for the LobbyClient. When connecting to the server, it will display a nice "Connecting..." text and when handshaking between client/server has been completed, it will show the same text as the lobby(ie, n out of m connected players).

Now, let's dive into the messaging system:

I decided to cut up the system into small, manageable components. This means there's a Server, LobbyServer and GameServer. The Server object manages the connections and routes the messages to the sub-system. The responsibility of the LobbyServer / GameServer should be obvious.

Because I wanted to seperate out the systems, the first byte in a message is the 'targeted' sub-system. The 2nd byte is the message and the 3rd and beyond bytes are message specific:
1 byte 1 byte n bytes
[Sub-System] [Message] [Message Specific Data]

There are no fields in my messages which specify the length of data. Because Lidgren handles all the low-level stuff, it will wrap my data into UDP packets, and reading data back is as simple as msg.ReadString() / msg.ReadByte(). No need to check for correct length, or if all the data is valid: Lidgren does that terrible work for me.

The pro of my system, is that I don't get insanely long switch statements to handle all the messages. This is how my 'main' message pump looks on the server:


It's quite simple, and adding new sub-systems is a matter of seconds.

The ProcessMessage() then reads the next byte in the message, which contains the message, which is used to pass it to the correct handler. That looks like this:


How each message is composed isn't really that important. I can give you the exact details, but it sounds more boring than interesting.

We get signal!

Hmmm, I think I used this title before… Not sure. And who cares anyway? It’s Sunday evening 22:45 now, almost time to get some shut eye before heading off to work.

Before diving into my bed, I’ll write up some nice post on my progress on the Dune Networking Components, and post it when my internet connection is back up again.

At the moment, I can connect clients to the in-game lobby. It will accept their connection, and once it receives a handshake, it will return their player ID, and a list of all connected players. This needs to be expanded with a faction and a color, but that isn’t in yet.

This is how the Lobby looks like server side:

Looks pretty sweet eh(I think I broke my FPS counter… There is no way in hell that runs at 35 FPS)? Well, it’s far from done, but I’ll get to that later on. This is what the basic lobby still needs:

• Handling of disconnects
• Player colors
• Player factions

There are more things required for the lobby, such as map selection, game options(Starting credits, starting units, how do we handle starting points(fixed / random), spice blooms, etc.). But for the sake of simplicity, I don't add those in. It would only complicate things while I want to get basic networking up and running.

Don’t get me started on the client side… That one just connects and sends the handshake. Nothing else yet. That's next up.

For the networking library I’m using the Lidgren Network Library. It’s pretty awesome if you’re working in .NET. It’s extremely easy to use and does all sorts of things for you, such as internal clock synch, message re-sending, etc. Speaking of which, I should see if there's a new version out the library. The version I'm using is a few months old.

No technical details to explain to you guys yet, so you have to wait a bit longer. The cool technical details come when I start working on synchronizing the simulations.

Let's get connected!

It’s been a good month(!!!) since I did a update on the devlog. So I figured I’d give you guys a new update on how things are going. The updates are going in a bit weird way this time, because I made an oopsie with my internet bill(I accidently forgot to pay a month, and they put a nice “Please pay us and you get internet access again kthx” banner whenever I browse to any website.

That’s the annoyance with all the moving around, you miss a bill and then they shut you off. However, they put a filter on my line at Friday afternoon 18:30, 30 minutes after closing their billing department. If they did it an hour earlier, I’d fix them up the bill immediately and there wouldn’t have been a problem.

But, this is good news for Dune. As I can’t waste my time on the internet, I decided to start coding up the network components for Dune.

Design

As opposed to earlier posts, I decided to actually go with a Client / Server setup instead of a star setup. The P2P star is difficult to set up and I couldn’t figure out a way to be 100% be sure all clients would have the correct amount of data to make sure they don’t go out of synch.

Downside in my current approach is that the host will have to have at least n times as much bandwidth as regular players, where n is the amount of players – 1. But, in a time of broadband, I doubt that’ll be a problem.

I still intend to go with the turn-based approach: Each set of commands is send 1 (or 2) turns ahead of the current one. Turns are executed at a determined interval(for instance, 200ms) on authority of the server. So, when turn 1 is running, and the server knows all the clients have received the commands required to execute turn 2, it’ll send a StartTurn 2. However, this will mean, that if 1 client has a roundtrip time above the turn-length, the game will start to lag at some point. In order to prevent this, I intend to add some form for heuristic to automatically adapt the turn-time based on client pings. However, this won’t do magic, because if pings are above 300 ~ 400ms, turns will take too long and players will notice lag between commanding a unit and actually executing it.

Implementation

At the moment, I’m implementing the basic server / client code. The server runs as a Task in the game, as it needs to be updated at least once per frame. Next to it the client runs its normal simulation at the moment.

I’m still doubting if I give the server its own simulation: That would mean the hosting player would run the game twice: Its own copy, and a server copy. On slow machines, this would mean it would lag(Or to be more precise, run at low FPS).

However, I could hand the Server task a reference to the client simulation and use that as a base. It could generate checksums of that and request checksums from other clients to compare and kick out-of-synch players. But, that’s too far ahead.

I intend to write the code that hosts a basic lobby and let’s other clients connect. No graphical goodness tho, just basic command line stuff like:
dune2.exe –host 2 –name or dune2.exe –join -name

Once a client can connect to the server I can add the code to start a game and just show the same scenario on both game instances. I think that’s a challenge on its own already.

Updates!

So, let's explain what I'm up to all this time.

Personal situation

Last summer I landed a job as System Administrator at a company. The company itself had about 150 employees, and my job was assisting the sitting sys. admin to keep the servers up and running, fixing issues, etc. While the job was fun at the start, it become pretty boring in the long run. Esp. when the main sys. admin was on vacation, I had almost nothing to do. But, I needed the money to pay for college coming year. This was one of the reasons I stopped development on Dune: I came home from work after around 18:30, had to make dinner, eat and clean up again. By the time everything was done, the last thing I wanted to do was code.

Eventually, when classes started again, I worked there for 2 days a week, as it was a nice solid income. However, after a few weeks, I got an e-mail from a professor, with a job pamphlet. An acquaintance of the prof was looking for a student who wanted to do some C++ / OpenGL development on an existing CAM software package. So I called the guy, went in for a job interview and got hired.

So, right now, I'm working 2 days in the week as C++ programmer on CAM software. The software itself loads a CAD model on which the user can then do several operations to prepare the model to be cut out in wood, plastic or metal. Editing the model has to be done in CAD software, so the transformations are mostly simple ones. A pro of this job is that it pays 50% more of my old job, which is even better.

I also have a load of school assignments to do, which have a higher priority than Dune has.

Dune 2: The remake

I never intended to permanently cancel the project, I just didn't have any time to work on it. The downside of hobby projects eh? Anyway, I intend to continue development in the coming months and pick up where I left off: The networking engine.

Last night, I was considering my options on what I could do to get it working across the internet. Initially, I intended to go with a Star Topology network using UDP. Each client would connect over TCP to the host, which would then assign a port number to the client. Using NAT Punchthrough, the client would then connect to the host over UDP over the assigned port, and disconnect. The other clients would then connect to the client as the port would still be open. However, this is quite complex, and I do not know if it works. Since UDP is connectionless, the UDP port should stay opened in a router for a short time span, even after "disconnecting". However, I would need to test this to verify that it actually does.

If it won't work, I'll to go with a traditional client/server approach. However, I still intend to go with the same approach as AOE, go with turns, etc. However, this would be a problem in a P2P setup, because how do I ensure that each player executes the same commands?

For instance, consider player 1 moving units 10 and 11. These orders will be send around the network to be executed for gameturn n. Each client will respond back to player 1 that they received the orders. But, how do the other clients know that all the other clients have received the order aswell? If one client does not have these orders, and all clients execute the turn, this client will run out of sync, and gets kicked. So, I need to work out a little bit on how we're going to do this. I could technically let each client send a hash / CRC of all received commands from the other clients to the 'host', which then determines which commands get executed and which get re-scheduled for the next turn.

As you see, lots of thinking to do. I should make a few diagrams of this, as it's quite confusing in words only.

More info tomorrow!

I'm writing this in my bathrobe as I'm preparing for bed, but I got quite some ideas. I intend to work these out a little bit more tomorrow(Technically today, but it's tomorrow since I haven't slept yet :P) in a blog post, and try to examine the different possible solutions. Especially geared towards the networking engine.

I shall also try to explain my current personal situation and the amount of time I have at hand to work on the project, because I also need work 16 hours a week, and have class.

On a sidenote, someone commented that I used the term "fans", and said some of them are interested onlookers. I never meant this in the way of, "wooo, I got sum fans lol!", just as a little joke / term to described people interested in the project. But, the onlookers and die-hard Toolmaker fans(The ones having my name tatoo'd all over their body), rest assured, I shall try to work on the game a bit :). Good night for now.

Not dead yet!

Sorry guys! I have been really busy with a lot of things, so I couldn't spend any time working on Dune.

Last summer I worked the entire summer(I needed the money to get through this college year) and when I got home, I didn't really feel like coding anymore, even tho it was a system administrator job. After summer vacation was over I dived into school and got covered in work.

I recently started a new job as C++ programmer, for 2 days in the week besides to school. I didn't do a lot of coding for the past few months, so perhaps this will get me rolling again. I feel like coding a bit on Dune later this week. I don't have an awful lot to do this week.

Again, sorry for the lack of updates and the lack of progress. I know my "fans" are watching :D

Networking

I've started to look into the networking code for Dune. While I'm currently still in a phase of trying things out in a sandbox environment, I will work on the networking code from now on. Once that's in, I will add the other missing features in Dune.

I initially read a few articles on other (commercial) RTS games and how they did their networking. For instance, Age Of Empires used a Peer-to-peer setup to communicate with all the other players. The game was a perfect simulation on each client. They accomplished this by make the game 'turn-based'. The player input would be registered, packed up and send to all clients. Each action would be executed in a turn, and when the data was send out to all the other clients, it used current turn + 2. So, when the current turn was 1000, and the player wanted a villager to go and chop down a tree, the actual ordering would happen in turn 1002. Since turns only lasted 200ms each, the player wouldn't even notice the delay between clicking and seeing the action.

However, in such systems, it's problematic when lag is 500ms: A packet send with orders scheduled for turn 1002 would arrive during turn 1003. AOE would adopt to this by either halt the game(In severe cases), or stretch up the time needed for each turn. In the case of 500ms lag, it would increase the turn time to 250 or 275ms, giving each packet more time to arrive.

This seemed to work well(I tried AOE2 to test it out) and it ran like a charm. So, I wanted to implement a similar to that of AOE2. However, setting up a P2P network between clients is a pain in the ass. Esp. when you take NAT into consideration. The system would have to work like this:
- Each client connects to host
- On start game, the host opens a series of new ports, one for each client.
- Each client connects to this port and disconnects directly
For each client:
- The host sends all other clients to which other client they should connect by handing out the IP and port number.
- The clients connect to the other player since the NAT port is now opened up

- Host is done when all players are connected to eachother.

It's quite complex, and I'm not sure if it would even work. Perhaps I should give it another shot and try to set it up. Once it works, it should work like a charm.

Bugtracker update #2

It's been a while since I updated... I've been really busy. Since the summer vacation begun, I found myself a summer job. Right now I'm a system administrator at a large company. My job is to check out how Windows Vista will fit in the existing environment and how the group policies behave in them, since Vista doesn't eat the older formats. I also have to take over the responsibilities of the other system administrator when he's off on vacation.

I just continued with experimenting with bug trackers. I tried to install Trac, and failed. The reason: It seems it can create an sqlite database, but no matter where I try to create it, it just fails. If I create it in the root of the C drive, it fails with the error Access Denied, and if I use an existing directory, it screams that I can't create a new directory(Well duh!). So I gave up.

One of the biggest downsides of Trac is that it requires quite a lot of external dependencies. In order to run, you either need a running Apache configuration with CGI / FastCGI / mod_python installed. You need sqlite(Since MySQL is very experimental. And to be honest, I don't like to run experimental stuff on my development environment).

So: Bugzilla is left. I've decided not to install / try any paid trackers, since those are outside my budget anyway.

One a sidenote: I did have some more time with Mantis this week. Even tho it lacks SVN integration, it's still a pretty nice system. I like it, and if Bugzilla doesn't behave better, I'll go with Mantis and setup a SVN integration system. I can't remember the name right now, but it's capable of glueing Mantis / Bugzilla to SVN.

Bugtracker update #1

I haven't done a whole lot of research on bugtrackers in the past days, but that's because I've been doing some other things aswell(Like snowboarding the entire day. Yes, in the middle of the summer.). However, I can do some reporting on things I've done.

First of all, I installed Mantis. Since I already had an instance of Apache running, I could just install PHP for it, install MySQL and voila, done. After that I added a few bugs to the system to see how it works and looks.

A few downsides of Mantis:

• Documentation on the site is lacking
• The My View is kinda cluttered
• Not possible to assign a bug directly to a developer while reporting it(Actually, you can, but you need an advanced report for that)

But, it also has a few nice features:

• Easy to install(Just open the index page)
• Detailed report information
• Multiple projects in 1 database
• Projects can be made (in)visible to certain users, users can have roles in projects

So far, I like Mantis. But, I still have to look into a glue system to glue it into SVN(Which is out there), since it's pretty nice to be able to link repos updates directly to bugs.

I also made a start with installing Trac, but that requires quite some more work...

Counting the credits

I implemented the credits counter today. It's a simple little UI component that I managed to whip up in about 1.5 hours, including sexy 'animation'. After setting the credits, it compares the amount with the visible credits. In then calculates the difference between the 2 and divides that by 2(seconds). During each frame update, it multiplies the pre-calculated value with the deltatime and voila, you get to see an update animation.

It's not exactly like the original, that seemed to rotate around, but that's too much work to implement for now.

The credits counter in the top right

I'm also looking into using a bugtracker. Right now I'm using the SVN commit logs to document bugs, but that's not the most efficient way. So using a bugtracker might help with bugs that I intend to fix on a later date. As for now, I keep forgetting to document all the bugs, remember them AFTER the commit and write them down. And then lose the note with the bugs.

So, I'm looking at Mantis and a few others, and I'll make my decision soon(probably tomorrow). After I fixed my server... Yes, fixed... I decided to remove a bunch of server roles which I installed for testing purposes. However, after I removed the Terminal Server role it had to reboot. After the reboot I could no longer login remotely, so I'm trying to fix that now.

Carry-alls and harvesters... An excellent combination

Today I added in all the goodness for carry-alls to pickup harvesters and deliver them into the refinery. It was quite some work to get that working, since I had to code in a few extra special cases(is this a harvester, does it have clearance for the refinery, is the landing area actually the refinery, etc.). But, now harvesters DO get picked up by a carry-all, if around, and they'll fly em into the refinery. And flying is SO much faster than driving :).

Carry-all just picked up harvester enroute to spice field.

Also, a newly built refinery now will receive a free harvester, which is dropped off in style by a carry-all, which will depart immediatly after dropping off the harvester. After the carry-all leaves the onscreen boundaries, it will delete itself.

Fully loaded harvester being returned to refinery

The carry-all which delivers the reinforcements into the field is a little special carry-all, namely a ReinforcementCarryall. I wrote a special class for it, since I didn't want to implement the code for leaving the field in the regular carry-all. That would really pollute the code. Also, this new Carry-all makes it possible to have regular reinforcement drops too, which is pretty sweet. Including stuffing 20 units into it.

But, there's a little issue with the carry-alls right now: When a tile is used, it won't be able to drop off it's payload, and just keeps hanging there, until the tile becomes free. So, that's what I'm going to solve next: find a free tile and unload there.

I already wrote a similar thing for the harvester to find spice tiles, so I'm considering to make it a general helper class and use a generic Comparable kind of thing. It's code that won't be called too often so speed isn't a really big issue here. And added goodness is that I can 'smear' out the search over multiple frames. In the case of finding a spice field which needs 200 steps to be found, it will cause severe lag. Being able to do this is say, 40 frames, it will only be 5 steps per frame, and therefor no big deal.

After this works, I'm going to fix a little bug with the selection rendering, constructing and then finally implement the credit counter control.

Spice must flow!

And flow it will! I spent the weekend implementing harvesters and refineries. Right now, a harvester is capable to driving to a spice field, happily seperate the spice from the sand, find a refinery, drive back and enter the refinery. Once in, the refinery will empty the harvester, provide credits to the player and when the harvester is empty, it will deploy it.

Our initial state: Idle harvester and refinery

Most of the harvesting code is still the same as it was before, but I also changed quite a lot. Most of the changes are in the refinery, which now has a 'landing zone', the area where the harvester has to land in order for the spice to flow. Initially it used the building coordinates to find it's way into the refinery, but this meant the harvester would drive to the center of the building. Also, I updated the images concering the harvester and the animation, so they use up less VRAM(Altho it came with a dirty hack: I have to figure out in which directions to displace the animation, otherwise it looks dislocated)

The harvester is collecting spice

When a harvester is full, it will request a list of all known refineries. It will traverse this list and attempt to find the most suitable refinery. That is, with the shortest queue or no queue at all. When it comes by an idle refinery, it will stop searching. When it has found a refinery, it will inform the refinery that it's enroute, lay out a path and start it's dangerous journey back home(Esp. when no carry-alls are around). The refinery will now start 'blinking' it's landing zone(when idle), or otherwise just queue up the harvester.

Harvester returning to the refinery, which has a blinking landing pad

When the harvester has reached the refinery, it will wait for clearance. The clearance means the refinery is ready to receive the harvester and refine whatever goodness it brings. When the harvester receives it's clearance, it will roll onto the landing zone and remove itself from the scene. This is important, because it will otherwise keep rendering itself, which will look weird: A harvester ontop of a refinery with a harvester in it. The refinery stores the harvester internally, so when the refinery is destroyed, it will automatically destroy the harvester aswell.

Refinery processing a harvester

It takes 35 seconds for a fully loaded harvester to unload(20 credits per second). When done, a harvester will ALWAYS be deployed, even when a carry-all is signaled to pick it up. This is done because it might take a while before the carry-all arrives, and when deployed, it gives other harvesters the chance to enter the refinery. This was something that annoyed me in the original game: When a harvester was waiting for a carry-all to arrive, it took a while, so other harvesters had to wait, while I was already low on cash.

To do:
- Have harvesters return back to the spice field
- Limit the number of tiles searched for new spice
- Harvester spice dunes first, then normal spice
- Fix bug related to tiles not being free'd
- Have a carry-all unload the harvester into the refinery
- Have carry-alls bring a harvester to a newly placed refinery

Epic debug sessions

I'm currently busy with some epic debug to fix a some major bugs in the movement code. It suffered from units running over other units, where the unit that was being ran over disappeared. It still rendered, but it no long accepted orders, could be selected, etc. I also had units that would not clear tiles they occupied properly, so that in some cases, tiles were still used while the unit was half a map further, which led to other units behaving strangely when driving. And last, units sometimes 'jumped' to other tiles and started vibrating. Yes, vibrating... No idea what causes that.

I fixed in the past week the run overs, which is nice. By fixing this, I haven't seen units warp since I fixed that, nor vibrate. So it seems that I only need to fix the clean up. I made a nice function which runs a check on the entire map and shows which tiles are used by what unit, and if this unit is actually positioned on that tile. And if not, it shows a [ERROR] tag. In combination with the debug log I hope to find out where it goes wrong.

After this works, I'm gonna make units 'defend' themselfs from others. ie, if attacked, they'll start attacking their attacker.