Improving Unsupervised Constituency Parsing via Maximizing Semantic Information.

Chen, Junjie, He, Xiangheng, Miyao, Yusuke and Bollegala, Danushka ORCID: 0000-0003-4476-7003 (2025) Improving Unsupervised Constituency Parsing via Maximizing Semantic Information. In: International Conference on Learning Representations (ICLR), 2025-4-24 - 2025-4-28, Singapore.

Text Improving_Unsupervised_Constituency_Parsing_via_Maximizing_Semantic_Information.pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (5MB) | Preview

Abstract

Unsupervised constituency parsers organize phrases within a sentence into a tree-shaped syntactic constituent structure that reflects the organization of sentence semantics. However, the traditional objective of maximizing sentence log-likelihood (LL) does not explicitly account for the close relationship between the constituent structure and the semantics, resulting in a weak correlation between LL values and parsing accuracy. In this paper, we introduce a novel semantic information (SemInfo) maximization objective that trains parsers by maximizing the semantic information carried by predicted structures. We introduce a bag-of-substrings model to represent the semantics and derive the SemInfo value using the probability-weighted information metric. We apply the SemInfo maximization objective to training Probabilistic Context-Free Grammar (PCFG) parsers. Additionally, we develop a Tree Conditional Random Field (TreeCRF)-based model to apply mean-field SemInfo maximization training to Probabilistic Context-Free Grammar (PCFG) parsers. Experiments demonstrate that SemInfo correlates more strongly with parsing accuracy than LL. Our algorithm significantly enhances parsing accuracy by an average of 7.85 points across five PCFG variants and in four languages, achieving state-of-the-art level results in three of the four languages.

Item Type:	Conference Item (Unspecified)
Divisions:	Faculty of Science & Engineering Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	14 Feb 2025 08:30
Last Modified:	06 Jun 2025 21:43
Related Websites:	Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3190280
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.