Performance Evaluation Of Supervised Machine Learning Classifiers For Mapping Natural Language Text To Entity Relationship Models
Article Sidebar
-
Entity Relationship Model, Information Extraction, Machine Learning, Machine Learning Classifiers, Natural Language Text
Abstract
Transforming natural language requirements into entities involves a thorough study of natural language text. Sometimes mistakes are made by designers when manually performing this transformation. Often, the process is time-consuming and inaccurate. Hence, multiple research studies have been performed to assist inexperienced designers in mapping a natural language text into entities and reducing the time and error that such a method entails. This work is part of those studies. Human intervention is a significant constraint for prior studies. In this paper, machine learning classifiers are used to eliminate human intervention. The system performs well in predicting entities and has achieved 85%, 75%, and 80% for recall, precision, and the F-score, respectively. The system also performs well in predicting nouns that do not represent entities and has achieved 68%, 79%, and 76% for recall, precision, and the F-score, respectively. The performance level of the system is the same as other model generation tools found in the literature. The system is distinguished from these tools in using machine learning classifiers as a technique for establishing entities with no human intervention. Furthermore, the study finds that when distinguishing entities from other nouns, logic-based classifiers, perceptron-based classifiers, and SVM classifiers perform better than statistical learning classifiers. The decision tree classifier, neural network classifier, and SVM classifier all work well. The decision tree is better because it can provide a decision tree that defines when a noun is an entity and when it is not based on given features; this is not the case with the neural network classifier and SVM classifier.
Full text article
References
[1]- Song. I.-Y., Zhu. Y., Ceong, H., & Thonggoom, O, (2015). “Methodologies for Semiautomated Conceptual Data Modelling from Requirements,”. In 34th International Conference on Conceptual Modelling, Stockholm, Sweden, 18-31.
[2]- Chen, P. P. S. (1976). “The entity-relationship model-toward a unified view of data,” ACM Trans. Database Syst., 1(1), 9-36. doi: 10.1145/320434.320440.
[3]- Meyer, B. (1985). “On formalism in specifications,” IEEE Software, 1(2), 6-26.
[4]- Gomez, F., Segami, C., & Delaune, C. (1999). “A system for the semiautomatic generation of E-R models from natural language specifications,”. Data & Knowledge Engineering, 29(1), 5781.doi: https://doi.org/10.1016/S0169-023X(98)00032-9.
[5]- Buchholz, E., Cyriaks, H., Düsterhöft, A., Mehlan, H., & Thalheim, B. (1995). “Applying a natural language dialogue tool for designing databases,”. In Proceedings of the First International Workshop on Applications of Natural Language to Databases (NLDB), Versailles, France, 119-133.
[6]- Burg, J., & van de Riet, R. (1998). “Color-x: Using knowledge from wordnet for conceptual modelling,”. In C. Fellbaum & G. Miller (Eds.). “WordNet, An Electronic Lexical Database,” Cambridge, MA: MIT Press, 353-377.
[7]- Du, S. (2008). “On the use of natural language processing for automated conceptual data modelling (PhD thesis),” University of Pittsburgh. Retrieved from http://dscholarship.pitt.edu/8965/1/du-siqing.pdf.
[8]- Harmain, H. M., & Gaizauskas, R. (2003). “CM-Builder: A Natural Language-Based CASE Tool for Object-Oriented Analysis,” Automated Software Engineering, 10(2), 157-181. doi:10.1023/A:1022916028950.
[9]- Meziane, F., & Vadera, S. (2004). “Obtaining ER diagrams semi automatically from natural language specifications,” In Sixth International Conference on Enterprise Information Systems (ICEIS 2004). Porto, Portugal, 638-642.
[10]- Kim, Y. M., & Lee, T. H. (2020). Korean clinical entity recognition from diagnosis text using BERT. BMC Medical Informatics and Decision Making, 20(7), 1-9.
[11]- Omar, N., Hanna, J. R. P, & McKevitt, P. (2004). “Heuristic-based entity-relationship modelling through natural language processing,” In Proc. of the 15th Artificial Intelligence and Cognitive Science Conference (AICS), Galway-Mayo Institute of Technology (GMIT), Castlebar, Ireland, 302-313.
[12]- Han, X., & Wang, L. (2020). A Novel Document-Level Relation Extraction Method Based on BERT and Entity Information. IEEE Access.
[13]- Tjoa, A. M., & Berger, L. (1994). “Transformation of requirement specifications expressed in natural language into an EER model,” In R. Elmasri, V. Kouramajian & B. Thalheim (Eds.), “Entity-Relationship Approach — ER '93,” Lecture Notes in Computer Science, 823, Berlin, Heidelberg: Springer, 206-217.
[14]- Tseng, F. S., Chen, A. L., & Yang, W.-P. (1992). “On mapping natural language constructs into relational algebra through ER representation,” Data & Knowledge Engineering, 9(1), 97118.
[15]- Athenikos, S. J., & Song, I. Y. (2013). “CAM: A Conceptual Modelling Framework based on the Analysis of Entity Classes and Association Types,” Journal of Database Management (JDM), 24(4), 51-80.
[16]- Ambriola, V., & Gervasi, V. (2006). “On the systematic analysis of natural language requirements with circe,”Automated Software Engineering, 13(1), 107-167.
[17]- Sugumaran, V., & Storey, V. C. (2002). “Ontologies for conceptual modelling: their creation, use, and management.,” Data & Knowledge Engineering, 42(3), 251-271. doi: https://doi.org/10.1016/S0169-023X(02)00048-4.
[18]- Herchi, H. & Abdessalem, W. B. (2012). “From user requirements to UML class diagram,” In International Conference on Computer Related Knowledge, Sousse, Tunisia. Retrieved from http://arxiv.org/abs/1211.0713.
[19]- Thonggoom, O., Song, I.-Y., & An, Y. (2011). “EIPW: A Knowledge-Based Database Modelling Tool,” In C. Salinesi & O. Pastor (Eds), “Advanced Information Systems Engineering Workshops,” CAiSE 2011. Lecture Notes in Business Information Processing, 83. Berlin, Heidelberg: Springer.
[20]- Thonggoom, O. (2011). “Semi-automatic Conceptual Data Modelling Using Entity and Relationship Instance Repositories (PhD thesis),” Drexel University, Philadelphia, PA, USA.
[21]- Chen, P. P. S. (1983). “English sentence structure and entity-relationship diagrams,” Information Sciences, 29(2), 127-149.
[22]- Hartmann, S., & Link, S. (2007). “English sentence structures and EER modelling,” In Proceedings of the fourth Asia-Pacific conference on conceptual modelling - Volume 67, Ballarat, Australia, 27-35.
[23]- Overmyer, S. P., Lavoie, B., & Rambow, O. (2001). “Conceptual modelling through linguistic analysis using LIDA,” In Proceedings of the 23rd international conference on Software engineering, Eden Roc Renaissance, Miami Beach, USA, 401-410.
[24]- Elbendak, M. E. (2011). “Requirements-driven Automatic Generation of Class Models (PhD thesis),” Northumbria Univeristy, Newcastle upon Tyne.
[25]- Omar, M., Abdulla, A. (2020). “The Entities Extraction for Entity Relationship Models from Natural Language Text via Machine Learning Algorithms,” In Proceedings of the 4th International Conference of Basic Science and Their Applications, Elbeida City, Libya.
[26]- Henrique, B. M., Sobreiro, V. A., & Kimura, H. (2019). Literature review: Machine learning techniques applied to financial market prediction. Expert Systems with Applications, 124, 226-251.
[27]- Kotsiantis, S. B., Zaharakis, I. D., & Pintelas, P. E. (2006). “Machine learning: a review of classification and combining techniques,” Artificial Intelligence Review, 26(3), 159-190.
[28]- Sen, P. C., Hajra, M., & Ghosh, M. (2020). Supervised classification algorithms in machine learning: A survey and review. In Emerging Technology in Modelling and Graphics (pp. 99-111). Springer, Singapore.
[29]- Al-Safadi, L. A. (2009). “Natural language processing for conceptual modelling,”. International Journal of Digital Content Technology and its Applications, 3(3), 47-59.
[30]- Suárez-Paniagua, V., Zavala, R. M. R., Segura-Bedmar, I., & Martínez, P. (2019). A two-stage deep learning approach for extracting entities and relationships from medical texts. Journal of biomedical informatics, 99, 103285.
[31]- Zhang, Z., Zhan, S., Zhang, H., & Li, X. (2020). Joint model of entity recognition and relation extraction based on artificial neural network. Journal of Ambient Intelligence and Humanized Computing, 1-9.
[32]- Liu, Jin, Yihe Yang, and Huihua He. "Multi-level semantic representation enhancement network for relationship extraction." Neurocomputing 403 (2020): 282-293.
Authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
In a brief statement, the rights relate to the publication and distribution of research published in the journal of the University of Sebha where authors who have published their articles in the journal of the university of Sebha should how they can use or distribute their articles. They reserve all their rights to the published works, such as (but not limited to) the following rights:
- Copyright and other property rights related to the article, such as patent rights.
- Research published in the journal of the University of Sebha and used in its future works, including lectures and books, the right to reproduce articles for their own purposes, and the right to self-archive their articles.
- The right to enter a separate article, or for a non-exclusive distribution of their article with an acknowledgment of its initial publication in the journal of Sebha University.
Privacy Statement The names and e-mail addresses entered on the Sabha University Journal site will be used for the aforementioned purposes only and for which they were used.