Yang and Yachi held a first meeting in November, a time when relations between the countries were at a low point. After that meeting, the two governments released a landmark four-point consensus intended to improve the strained ties between them.
A statement released after Thursday's meeting said both sides agreed that initiating the dialogue is "a major measure for both countries to enhance high-level strategic communication", and will help to achieve consensus and manage and control differences.
"It is hoped that the two sides will maintain political dialogue and the momentum of the improving Japan-China relationship," Yachi said.
Yachi is on a three-day visit to China, and leading Japanese media such as the San-kei Shimbum newspaper speculated that he will discuss a possible visit to China by the Japanese prime minister later this year.
Ruan Zongze, vice-president of the China Institute of International Studies, said that as the relationship thawed, the dialogue "will help to nurture the positive momentum of the improving ties" as a part of joint official efforts.
The dialogue will also "help contain the actions of Japanese right-wingers" at a time when the country's Cabinet is creating uncertainty over its position regarding the history issue, Ruan said.