Smooth Coastlines in One Layer

[G|I|Sandy]

This technique is based on layer styles first suggested by _Capper_ in his tutorial. I attempted to perfect his technique. With this method, there will be no sea or land clipping at all. The best thing, once you set the layer styles, you do not have to do anything, just fill the land with black (5 5 5) and it will take care of everything (in fact, even better than if you do it manually). The only disadvantage is that coasts tend to be too round with this approach. So you may need to have some irregularity in your coasts.



Here i use one land layer (5 5 5 fill) above and one sea layer (0 0 253 fill) below. Heights (whiter areas) are optional. The land layer has 3 styles applied (see the settings below). The settings have been optimized for 5 5 5 and 0 0 253 colors. There are 2 custom gradients that you will need to set up yourself. These gradients are crucial to get smooth transition between colors 1 pixel inside land (~1 1 1 to ~5 5 5) and 1 pixel outside in sea (~0 0 235 to ~0 0 253). Playing around with settings can produce different results, though i believe deviating too much from the current settings will lead to land and sea clipping onto each other.

Smooth Coastlines in One Layer

Jpeg has messed up the colors. Attached is a sample .psd file just in case.

Sandy

[Gigantus]

That sounds promising. Do I understand that I will have to set the 'coastal' pixel at RGB 5 5 5 like this (no diagonal gaps)?:

xxxx   xxx
   xxxxx

[G|I|Sandy]

There is no need for any manual painting of coasts. It can be a flat 5 5 5 for the entire land mass. Even mountains right next to sea are fine since one of the layer styles introduces inner shadow of 1-2 px with a gradient. This ensures that you have 1 px of coast that is not too high. I simply filled the entire land mass with 5 5 5, then painted mountains with a soft brush at low opacity, just like you would normally do. This is why i feel this is the best method. There is nothing to do manually, it takes care of everything. Also the gradients are applied not inwards, but alongside the coast for both land and sea - hence the output is very smooth and curvy.

To apply the exact same styles right click on styles of "land" layer of the attached .psd, then right click and paste style into your own landmass layer.

Sandy

[Gigantus]

I was thinking of applying it to an existing map, hence my question. Unless I misunderstood the procedure.

Maybe a simple step by step method, eg how to create the layers (from what) and how to apply the result? In capper's tutorial I created the layers from the map_heights graphic, but I left the landmass as it was.

[G|I|Sandy]

The technique requires the landmass of any shape and height. The base height should be something above 0 0 0. I suppose any normal height between 1 1 1 and 8 8 8 (commonly used in mods i presume) should work. How you get the landmass is not important. I am not sure if you could use satellite images or anything like that. It would require some work i guess (like with any other method). In my case i simply painted the landmass from a physical map of England. Then applied the style shown in the images.

In case of applying to an existing heights map, i would suggest to use brightness/contrast tool to brighten up dark pixels (0 0 0). Water would clip into 0 0 0 coasts too much. Ideally, the darkest pixels should be above 0 0 0 (5 5 5 works well). In case of 2 layer styles by Capper, i would suggest to remove the styles and merge the two layers, then apply the new styles.

Sandy

[againster]

thanks! sandy

it works better than the method i used before

[Gigantus]

The technique requires the landmass of any shape and height. The base height should be something above 0 0 0. I suppose any normal height between 1 1 1 and 8 8 8 (commonly used in mods i presume) should work. How you get the landmass is not important. I am not sure if you could use satellite images or anything like that. It would require some work i guess (like with any other method). In my case i simply painted the landmass from a physical map of England. Then applied the style shown in the images.

In case of applying to an existing heights map, i would suggest to use brightness/contrast tool to brighten up dark pixels (0 0 0). Water would clip into 0 0 0 coasts too much. Ideally, the darkest pixels should be above 0 0 0 (5 5 5 works well). In case of 2 layer styles by Capper, i would suggest to remove the styles and merge the two layers, then apply the new styles.

Sandy

I'll give it a shot (I am a bit PS handicapped) - map_heights should never have an RGB value of 0 0 0 as that gets generated as water.

[Withwnar]

Nice, thanks for sharing.

Jpeg has messed up the colors.

Perhaps a PNG would be better?

[Gigantus]

Heureka! Even I managed to get it right - 99% of sea\land triangles are gone. One little spot left.

For some weird reason it invalidated one port placement, but changing the sea pixels right next to it to the old RGB value fixed it.

[Frunk]

Can someone explain the steps in greater detail? I am a PS beginner. Looks great though!

[G|I|Sandy]

Frunk, open the attached psd and see the settings. If you can't figure the rest, then i am sorry. You would better google something like "how to use layers in photoshop" and "layer styles".

Sandy

[Yayattasa]

Great idea, will try it asap. I tried my hand in something like that, but I was working with C instead of just using the layers. Really more sophisticated and more practical too.

[Frunk]

Frunk, open the attached psd and see the settings. If you can't figure the rest, then i am sorry. You would better google something like "how to use layers in photoshop" and "layer styles".

Sandy

Indeed. Sorry to be a pain. I notice on the .psd that inner glow settings exist (but aren't applied). Should I apply them too, or is it personal preference?

Regardless, I took your advice, had a play around and have my first attempt completed. No result yet as it appears a few invalid settlements and ports have to be moved, and I'll get into fixing them tomorrow.

[Frunk]

Done! Not perfect, but a Hell of a lot better than I could have hoped to do. At first I had some really weird results, which I found was due to some sea pixels being altered to values like 1, 1, 238. This was easily fixed by magic-wanding them and replacing with 0, 0, 238. A screenshot is below for anyone curious.

Spoiler Alert, click show to read: 

Worked brilliantly for navigable rivers, too.

Before:

Spoiler Alert, click show to read: 

After:

Spoiler Alert, click show to read: 

For a process that took no more than 6 hours total, I'm pretty stoked with the results. Thanks again Sandy.

[G|I|Sandy]

Glad to see things worked out for you.

Sandy