In Indonesia, the communicable diseases have become a serious concern. However, these diseases still cannot be handled properly because of the limited awareness and preparedness in facing the disease threat. This is as a result of that most Indonesian people do not have the capability to read and understand the health surveillance data often presented in a graphical presentation (i.e chart image). Therefore, the chart interpreter system for interpreting the related data into the health surveillance summaries based on Indonesian language were developed using Natural Language Generation (NLG) approaches. However, the naturalness of summaries obtained from the intended systemis still limited. Besides, the related system still depends on the manual input from the experts. Therefore, for overcoming these problems, this study develops a semi-template based that can be implemented into data-to-text architecture. Semi-template based applies a number of possible sentence patterns to be the templates in generating the related summaries. The intended pattern is obtained from the example of the existing health surveillance summaries. The experiment result shows that this method can resolve those problems, such that the naturalness of summaries generated by our system is higher than the naturalness of summaries generated by Pratomo’s system. Besides, our system can work automatically in order to interpret one-line and two-lines chart.