Comparison Between Some Penalized Regression Methods in the Presence of Multicollinearity: An Applied Study on Chronic Renal Failure Patients
Article Sidebar
-
Elastic Net regression, Lasso Regression, Multicollinearity, Ridge Regression, Robust Ridge Regression
Abstract
This article discusses some penalty methods for addressing the problem of multicollinearity (Lasso, ridge, robust ridge, and elastic net), and compares them with the least squares method. Also, it relies on a simulation approach under varying degrees of multicollinearity. Ten independent variables were generated at different sample sizes ranging from 50 to 450 observations, and 50 independent variables were generated at a sample size of 1,000 observations to compare and demonstrate the best proposed methods based on the MSE criteria, the coefficient, and the adjusted coefficient of determination. This article also relies on real data on chronic kidney failure patients, collected from the Kidney Services Center in Al-Khums City (January to August 2023). The number of observations reached 100, and 34 were excluded due to unavailability of data. The data included 14 variables (age، hemoglobin، urea، potassium، phosphorus، calcium، protein، uric acid، magnesium، cholesterol، triglycerides، vitamin-D، glomerular filtration rate، and creatinine). Using R version 4.5.1, the results demonstrated the flexibility of the Lasso method in handling data with multiple correlations، and to be more efficient than the other studied methods when analyzing real data in various simulations across different sample sizes. Furthermore, the results of the real data analysis showed ، the GFR of chronic kidney disease patients is negatively affected by levels of creatinine, magnesium, age, phosphorus, protein, and cholesterol, while it is positively affected by levels of hemoglobin, uric acid, and vitamin D. Finally, the article recommends adopting the Lasso method as the preferred option when dealing with multicollinearity, especially when the data are free of outliers and the sample size is proportional to the number of variables, regardless of the degree of correlation between them, due to its ability to achieve a balance between accuracy and simplicity in the model.
Full text article
References
[1] الديب، نادية عبد الله محمد. 2023. استخدام طريقة انحدار الحافة لمعالجة مشكلة التعدد الخطي. رسالة ماجستير، الاكاديمية الليبية طرابلس-ليبيا -قسم العلوم الرياضية -شعبة الاحصاء.
[2] بسيوني، عبد الرحيم عوض. 2023. دراسة مقارنة لطرق علاج مشكلة الازدواج الخطي بالتطبيق على الهجرة الداخلية في مصر. مجلة التجارة والتمويل.
[3] حيدر، علي ناظم محمد. 2023. دراسة اختيار أفضل نموذج انحدار للجلطة الدماغية باستخدام بعض الطرائق الجزائية. رسالة ماجستير، جامعة القادسية، كلية الإدارة والاقتصاد، قسم الإحصاء.
[4] الحلواني، ماجي خليل. 2022. استخدام أسلوب انحدار ريدج لتقدير حجم الهجرة الداخلية في مصر. المجلة المصرية للتنمية والتخطيط.
[5] الغندور خالد محمد، والدواخلي، وائل سعد. 2021. استخدام بعض طرق المربعات الصغرى الجزائية لتقدير واختيار متغيرات نموذج الانحدار الخطى في ظل وجود التعدد الخطي “، المجلة العلمية للاقتصاد والتجارة، جامعة عين شمس.
[6] الكفيشي، سارة. 2020. تقدير معلمات أنموذج الانحدار الخطي المتعدد في ظل وجود مشكلة التعدد الخطي. مجلة الإدارة والاقتصاد جامعة كربلاء.
[7] A. R. Nur، A. K. Jaya، and S. Siswanto. 2024. "Comparative Analysis of Ridge، LASSO، and Elastic Net Regularization Approaches in Handling Multicollinearity for Infant Mortality Data in South Sulawesi," Jurnal Matematika، Statistika dan Komputasi، vol. 20، no. 2، pp. 311–319. doi: 10.20956/j.v20i2.31632.
[8] N. Herawati، A. Wijayanti، and A. Sutrisno. 2023. "The Performance of Ridge Regression، LASSO، and Elastic-Net in Controlling Multicollinearity: A Simulation and Application," Journal of Modern Applied Statistical Methods، Vol. 23، Issue 2 .
[9] Efiezomor، Rita. Obikimari. 2023. A comparative study of Methods of Remedying Multicollinearity. American Journal of Theoretical and Applied Statistics. 124:87–91.
[10] Fonti، V. 2017. Feature Selection using LASSO. Research paper in business Analytics، VU Amesterdam.
[11] Duzan، H.، & Shariff، M. 2015. Ridge Regression for Solving the Multicollinearity Problem Review of Methods & Models. Journal of Applied science.
[12] Hubert، M.، Rousseeuw، P. J.، & Verdonck، T. 2012. A deterministic algorithm for robust regression and outlier detection. Journal of Computational and Graphical Statistics، 213، 618-637.
[13] Hoerl، E.، & Kennard، R. W. 2000. Ridge Regression: Biased Estimation for Non-Orthogonal Problem. Technometric. 421: 80-86.
[14] Lawrnce، K. D. & Arthur، J. L. 1990. Robust Regression: Analysis & Application. Marcel Deker، New York.
[15] Zou، H.، & Hastie، T. 2005. Regularization and Variable Selection Via the Elastic Net. Journal of the Royal Statistical Society: Series B، 672: 301–320.
Authors
Copyright (c) 2026 Journal of Pure & Applied Sciences
This work is licensed under a Creative Commons Attribution 4.0 International License.
In a brief statement, the rights relate to the publication and distribution of research published in the journal of the University of Sebha where authors who have published their articles in the journal of the university of Sebha should how they can use or distribute their articles. They reserve all their rights to the published works, such as (but not limited to) the following rights:
- Copyright and other property rights related to the article, such as patent rights.
- Research published in the journal of the University of Sebha and used in its future works, including lectures and books, the right to reproduce articles for their own purposes, and the right to self-archive their articles.
- The right to enter a separate article, or for a non-exclusive distribution of their article with an acknowledgment of its initial publication in the journal of Sebha University.
Privacy Statement The names and e-mail addresses entered on the Sabha University Journal site will be used for the aforementioned purposes only and for which they were used.