I wouldn't be so sure about that. There was a huge difference between Japanese military strength when the first intervention by commodore Perry occurred, compared to the size of their military at the time of the Hawaiian coup. As Oda pointed out earlier, Japan was the premier naval power in the pacific at the time. It is far from inconceivable to believe that Japan could have come out on top if such a clash had occurred.
I think the winner would have been determined by when exactly such a conflict was to break out. If Japan seized the islands, crushed the coup and reinstated the Hawaiian monarchy in 1883, leading to open war between themselves and the USA, then it likely would not have gone in their favor. At that time Japan had not yet established military supremacy over China and as a result was very much concerned with their immediate neighbor.
On the other hand, if war had broken out in 1887 when the US officially annexed Hawaii, Japan would have been in a much stronger position. Their naval power had been greatly expanded after the stunning victory over China in 1885, not to mention all the important naval experience and lessons gained from that. The US military, like all western military's right up to ww2, were completely contemptuous of 'oriental' soldiers and I wouldn't be the slightest bit surprised if the US navy completely underestimated the Japanese the same way the Russians did.
Teddy Roosevelt was adamant that Japan had a stronger navy than the US pacific fleet and used this as a major argument for Hawaiian annexation as well as support for the panama canal project. The Japanese navy would almost certainly have crushed the Asiatic squadron and at the very least would have been an equal match for a US naval force sent from the west coast to capture Hawaii. Who knows, the Americans may even has sent their Atlantic fleet around south America and meet the same fate that the Russians did!
Also on the topic of the Russians, the Russians were still building up their pacific fleet 1887, with its massive port arthur naval base 'leased' from China still very much under construction and war would not break out between them for a further 8 years. As a result it would be safe to say the Japanese could have engaged in a conflict with the US over Hawaii without too much fear of a surprise Russian attack. Also the fact that the Russian far eastern Navy was riddled with Japanese spies would have made a surprise attack highly unlikely