Published Paper Details:

The analysis of software evolution in object-oriented systems often generates large volumes of code and documentation, making manual inspection challenging. Lexical feature extraction provides an effective way to derive insights from these software artifacts across multiple abstraction levels--from individual identifiers and comments to class- and project-level semantics. This paper introduces a novel multi-level framework for automated lexical feature extraction that combines natural language processing (NLP), static code analysis, and representation learning techniques. The framework captures lexical, syntactic, and semantic patterns across software versions, enabling better insights into software evolution. Evaluations conducted on several open-source repositories demonstrate that the framework improves key software maintenance tasks, such as change impact analysis, feature location, and code summarization. The results show that multi-level lexical features enhance model performance in evolution analysis by up to 23%, compared to traditional single-level approaches.