My Resume

The evolution of communication from the post-writing era to the telecommunication age has given rise to the digital writing register. Although this register accounts for a vast majority of daily communication, it has been largely neglected in Persian corpus linguistics due to its inherent processing challenges. By manually compiling and processing one of the most challenging sub-registers of this domain—microblogging—this dissertation aims to fill the existing gap in corpus availability and provide practical solutions to facilitate its processing. Our objective was to develop a microblog sub-register corpus that simplifies fundamental processing tasks while preserving linguistic variation and reducing the overall vocabulary size. To achieve this, we extracted a balanced, randomized dataset of one million tweets from the year 1400 AP (2021–2022), titled the "PT1400" (Persian Tweets of 1400). From this, a sub-corpus comprising over one million tokens was randomly selected for manual annotation. We employed a "distributed approach" for the manual processing phase, addressing challenges across normalization, tokenization, POS tagging, and lemmatization through task distribution and interaction between these sequential processes. For POS tagging, we introduced a novel and flexible framework called TagFrame to encode the tag sets. Beyond enabling the representation of linguistic and paralinguistic information and preserving morphological and orthographic variations, TagFrame allows us to represent the internal structure of words. Furthermore, we utilized trace-implementation as a new method to reduce the corpus vocabulary size, and employed diacritics during lemmatization to disambiguate homographs. Additional outcomes of this effort include comprehensive POS tagging guidelines, a re-examination of select Persian grammatical concepts, and the introduction of a standardized Persian TagFrame. The resulting benchmark corpus was manually normalized and tokenized, featuring 85 attributes encoded across 2,045 unique tags organized into three distinct layers: lexical, intra-lexical, and paralinguistic. Statistical and comparative analysis of PT1400 and its lexical categories against other corpora revealed several defining characteristics of the microblog register: high lexical diversity alongside a prominent frequency of single-occurrence tokens (hapax legomena), high frequency of verbs, pronouns, and interjections, low frequency of nouns, prepositions, conjunctions, and standard punctuation, distinctive usage of specialized categories, such as emojis and identifiers, overall message brevity and the presence of taboo words. These features are closely correlated with the situational variables of the participants, relations among participants, channel, communicative purposes, topics, and processing circumstances.

In this research, we studied the Persian inflectional tense forms and how they are processed. Since historical studies are dominant in the study of language in Persian, we dedicated the first two questions of this research to revise some of the concepts in the area. In the first question, we asked about the categorization of Persian verbs based on the concept of regularity. The results showed that there are three categories in Persian verbs: regular, irregular, and alternative. Regulars are an open category in which their past tenses are produced by adding the regular suffix (: -id) to root and don’t belong to the other categories. Irregulars are a close category which there are no systematic relation between their roots and past tense forms, and they are represented as a list of connected pairs; alternatives, which are a kind of regular verb, are an open category which doesn’t belong to irregulars, their roots consist of more than two syllables, end with -ɑn, and they use -d and -id suffixes to produce their past tense forms. In the second question, we asked about the model which represents the process of tense inflection. The results showed that the present tense of regulars and irregulars, in addition to the past tense of most of the regulars, are produced by retrieving the root and adding proper affixes. On the other hand, the past tense of irregulars and some of the high-frequency regulars are produced by retrieving the past stem from memory. In the third question of this research, we asked about the mental mechanisms that are involved in producing Persian verbs. Steven Pinker claims that the distinction between regulars and irregulars roots in two distinct mechanisms in which rule mechanism involves in producing the regulars and rote mechanism involves in producing irregulars. The results of this research confirm Pinker's claim. To examine these three questions we used a wide range of evidence, but the main source of the evidence came from an experiment in which we presented participants by the present tense of verbs and asked them to produce the past tense of these verbs as fast as possible. Then we gathered the errors that they made and used them as a source for our purposes.