A stunning left-footed volley from substitute Tadanari Lee broke the deadlock on 109 minutes and the Australians were unable to equalise and force penalty kicks.
Australia has now been dumped by Japan from its only two Asian Cup appearances, having lost on penalties at the quarter-finals stage in 2007.
The Japanese previously won the Asian Cup in 1992, 2000 and 2004.