Easy "work of art" coastlines

[Linyoa]

I've seen this discussed here a few times before, but, while offering great results, the methods were often complicated, or they were easy, but didn't do a very good job.

SigniferOne created a nice guide for instance
http://www.twcenter.net/forums/showthread.php?t=241854

but for those of you who want to save some time on large maps I'll show you my method:
First off you need two things:
Gimp
map_heights.tga
I suppose photoshop would work just as well, but there may be some differences I am not aware of.

Also I tried to make this a guide anyone could use even if you have never used gimp before so I included a ton of photos. If you are good at gimp simply don't open the spoilers.
This guide should remove almost all land and see triangles, but some may still be left so you may have to fiddle with it.

1. Open up map_heights.tga (yourmod/data/world/maps/base/map_heights.tga)

2. Using the "Select by Color" tool select all the bodies of water on your map.

Spoiler Alert, click show to read: 

3. Fill the body(ies) of water with 0,0,253
To do so simply click on the color selector and put in the RGB code of 0,0,253 then click ok

Spoiler Alert, click show to read: 

You should now have blue as your selected color (specifically 0,0,253 blue)
now select the bucket fill tool and select fill whole section

Spoiler Alert, click show to read: 

Oh, and make sure that the opacity is set to 100, and the mode is normal.
Now simply click anywhere within your selected region. This will fill it with 0,0,253

4.Now go to select, then grow, and choose grow selection by 1 and hit ok.
You should now also have the coasts selected

Spoiler Alert, click show to read: 

5.Now we need to deselect the bodies of water so we're going to use "select by color" again only this time hold control when you click on the water so that rather than selecting you deselect it.

Spoiler Alert, click show to read: 

You should now only have the coasts selected

6. Now we need to change the color of the coasts to 1,1,1
as in step 3 we are going to use the color selector to change the color to 1,1,1

Spoiler Alert, click show to read: 

7. Now fill in the entire coast with 1,1,1 using the bucket tool and fill whole section.
Again make sure you have normal mode selected with 100% opacity

Spoiler Alert, click show to read: 

8. Now every pixel adjacent to water should be 1,1,1. Good. Now for the time-consuming part :
Use the color selector to change the color to 2,2,2.
Now select the pencil tool. Make sure the mode is normal, the opacity is 100%, and you have brush: Circle (01) selected.

Spoiler Alert, click show to read: 

Now use the pencil to change each corner pixel to 2,2,2 (look at the picture to see what I mean. Red=2,2,2) This may take a while on larger maps.
(corner pixels are the pixels that only border water diagonally)

Spoiler Alert, click show to read: 

9. Now change the color to 0,0,0 and use the pencil select tool to change all the "opposite corners" (corners that are bordered on three sides by water) to 0,0,0

Spoiler Alert, click show to read: 

10. Good! Now save your work, delete map.rwm (yourmod/data/world/maps/base/map.rwm) and launch the game.

The results should look a little something like this:

Spoiler Alert, click show to read: 

Feel free to comment or ask questions
(but go easy on me, this is my first guide )

Thanks to:
Miraj
Gigantus
SigniferOne

For helping me figure this out

[The Bruce]

0,0,0 is not a valid land height.

Otherwise this is a good technique to get started and I use a similar method. However it results in very straight/jagged coasts.

[Linyoa]

0,0,0 is not valid in the sense that you cannot walk on it. I does however work fine for smoothing out coasts.
In any case you could change (increase) any of the land values so long as you decrease the corresponding sea values.

And it is true that this doesn't create totally smooth coast lines. For that you will need to read SigniferOne's Tutorial.
It is however good for removing land/sea triangles, and, in my humble opinion, significantly simpler and a good starting point.

[The Bruce]

0,0,0 = deep sea. It might look OK but ships will be able to sail on it. Isolated instances of 0,0,0 might be acceptable I suppose.

This approach is a very good starting point but why use 0,0,0 if other combinations work?

[Linyoa]

Actually, as far as I can tell (and as others have told me) 0,0,0 is counter intuitively very shallow water which is why it works to smooth out the coast line. I suppose I started with 0,0,0 so that this tutorial shows how to make the lowest coastlines possible. As I said you could really use any set of numbers so long as you set the land height proportional to the sea depth.

I may add a section in the coming days explaining how to make the whole coast or certain parts of it higher (I personally like around 10,10,10 as it gives the coast a sense of mass)

[wilddog]

I keep seeing more and more guides on this but its all pretty simple stuff.

Beats me why everyone keeps debating it. I simply used to run round the coasts changing the pixels to match what I wanted. You quite often use similar patterns. Not anything much more than that. The tricky bits were when you had very narrow sees or wanted to get a more precise shape.

[The Bruce]

The other thing people tend to ignore is that the height map is also used to generate battlemaps. For example having 10,10,10 coastlines might look nice on the stratmap but battles fought on the coast may have unrealistically uniform and steep coastlines.

It would be interesting to see how the battlemap generator interprets 0,0,0 pixels. On the stratmap these are certainly seen as sea tiles (without a .hgt file present). Whilst these pixels may not actually appear as deep sea on the stratmap I have no idea what they do to battlemaps. Regardless, ships can sail on 0,0,0 and that fact alone makes me conclude that this value should not be used. Why introduce bugs when you don't need to?

I agree with wilddog that there is no "quick" way to make "work of art" coastlines. To get the best results takes pixel by pixel manipulation. Also it is important to consider the effect on battlemaps especially if one is attempting a realistic simulation. For example, a small island near the coast on the stratmap can appear as a peninsula in battle.

The approach above is however a useful starting point but the use of 0,0,0 should be avoided.

[Linyoa]

It's true, I had not considered the effects on the battle map.
Hopefully 10,10,10 is not high enough to really be problematic but I will have to look into that.

On the other hand I'm not sure why you consider ships being able to sail on a water tile and men not being able to walk on it a bug.

As an example this is what that looks like:

Spoiler Alert, click show to read: 

The ship hugs the coast but it is not at all on land.

The only disadvantage of using 0,0,0 that I can think of is that it slightly decreases the size of your land masses. In normal situations this wouldn't be noticeable, but I could see it being a problem on very small maps or very small strips of land.

@Wilddog
It may be that I'm simply not the sharpest knife in the cupboard (actually that is highly probable) but I wasn't entirely sure how to do it after reading the two tutorials I found. I had to experiment to figure it out on my own (well, I had some help). And I also found that the tutorial I read was great for someone who had the time and patience to do it pixel by pixel and make it beautiful but were not so helpful for someone who simply wanted it about as good as it is in Vanilla (or simply remove sea triangles).
In any case an extra tutorial never hurt anyone

[wilddog]

@linyoa - no problem. I was simply trying to indicate that actually when you start to work out what you want to do its not that difficult to simply run around the coast bits and add the bits where necessary.

You are of course right adding any extra tutorial is useful and a plus ... SigniferOne put his together after he asked me how I did it. I don't normally get around to doing that sort of thing

I do tend to avoid presetting things to 1,1,1 or as someone recently posted 0,0,0 (I always got a flickering on and off on that setting) but its been awhile since I've done much mapping updates other than some occasional tweaks.

To be honest the hardest challenge on mapping was the original black rectangle issue which CA released a patch for that was far more significant (to me than the coastline bits).

[Linyoa]

really? I was almost sure that if used properly 0,0,0 is fine.
Well, its 2 on 1 so you guys a probably right.
I do agree that I wouldn't really recommend 1,1,1
(I used 1,1,1 simply as an example)

This weekend ill make some changes including a note on what you would need to do differently with different heights

[makanyane]

if you use 0,0,0 as 'land' value on an odd/odd square ie. the ones that control the land/sea difference then you get the alternating land/sea flashing effect on campaign map... and also probably add the possibility of boats being 'grounded' in those areas, I think that was a known bug....

you can use 0,0,0 to smooth coastlines if it's not on the odd/odd squares at which points it acts a bit like having a deep sea colour at that point - however if you do that you can be stuck with unadjustable triangles produced from high land behind as you then have no way of adjusting the transition with the actual land pixels behind the 0,0,0 pixel (I think it also did some undesirable stuff to the battlemaps) so 0,0,0 is generally best avoided.

there was some confusion in earlier games as the vanilla RTW maps used 0,0,0 for some coastal regions but that only worked if the map_heights.tga was converted to map_heights.hgt - removing the map_heights.hgt showed up the same problems as you get with using it in M2