I'm open to correction, these are just my current beliefs..
Java as a language is easier to fully know, but it's much more complicated to write a lot of things in due some rules like "everything is a class". I am curious, have you done much in C#? It's very different from Java, the main similarity is automatic memory management and much of the basic syntax.
I'd say Java has three pros, but the most important one for me is a big con.
- Severe language restrictions so there are fewer keywords and less unique language constructs. This makes it easier to know all the features, but harder to do a lot of the programming.
- "Portability" Not much to say here, C# is for Windows (and Linux with mono), Java isn't tied to anything.
- Javadoc is something I quite like, C#'s documentation isn't as easy to extract from the code.
I mostly do windows programming, and I use php/html/js for web stuff. It's a simple statistic that currently the vast majority of end users are on windows. If that changes I'll switch to Java if need be.
In the end I think you're right. Java is better to start out with since there's fewer unique things to learn, and it's good to be able to solve any problem without relying on special language constructs. But then once you understand programming, C# is much nicer for when you want to make elegant software for people to actually use, and make it sooner and easier. Assuming you're on windows.