Metode regularizacije s uključenim domenskim znanjem stručnjaka za odabir značajki u linearnom regresijskom modelu

doctoral thesis

• Title (english): Regularization methods that include experts’ domain knowledge for feature selection in a linear regression model
• Author: Marinela Mokriš; MBZ: 372281
• Mentor: Diana Šimić (mentor); MBZ: 125756
• Mentor: Dario Šebalj (komentor); MBZ: 344280
• Committee member: Jasminka Dobša (predsjednik povjerenstva); MBZ: 218640
• Committee member: Petra Žugec (član povjerenstva); MBZ: 319436
• Committee member: Ana Meštrović (član povjerenstva); MBZ: 250171
• Granter: University of Zagreb; Faculty of Organization and Informatics; Varaždin
• Defense date and country: 2025-02-17, Croatia
• Scientific / art field, discipline and subdiscipline: SOCIAL SCIENCES; Information and Communication Sciences
• Universal decimal classification (UDC): 311 - Statistics

Abstract

Ovo istraživanje imalo je cilj razviti metodu za odabir znaˇcajki u linearnom regresijskom mo- delu koja u regularizaciju ukljuˇcuje domensko znanje struˇcnjaka o utjecaju ulaznih znaˇcajki na izlaznu, pri ˇcemu se razlikovalo dva naˇcina ukljuˇcivanja znanja struˇcnjaka – deterministiˇcki i metaheuristiˇcki. Predložena metoda se evaluirala na podacima o troškovima energije i karak- teristikama zgrada javnog sektora Republike Hrvatske te na tri scenarija simuliranih podataka iz literature. Za... More evaluaciju su se koristile mjere uspješnosti predikcije (korigirani koeficijent determinacije, RMSE, MAE i SMAPE) te, u sluˇcaju simuliranih podataka, razlika izme ¯du stvar- nih i procijenjenih regresijskih koeficijenata. Metoda je uspore ¯dena s regularizacijom koja ne ukljuˇcuje domensko znanje struˇcnjaka te s regresijskim stablima. Rezultati na stvarnim poda- cima pokazuju da je uvo ¯denje domenskog znanja struˇcnjaka metaheuristiˇckim pristupom re- zultiralo poboljšanjem uspješnosti predikcije u odnosu na spomenute metode, dok je uvo ¯denje domenskog znanja struˇcnjaka deterministiˇckim pristupom u nekim sluˇcajevima rezultiralo po- boljšanjem uspješnosti predikcije. U okviru simulacijskog eksperimenta, uvo ¯denje domenskog znanja struˇcnjaka rezultiralo je poboljšanjem nekih mjera uspješnosti predikcije u odnosu na regularizaciju koja ne koristi znanje struˇcnjaka samo kod metaheuristiˇckog pristupa, dok pred- ložena metoda uvo ¯denja domenskog znanja struˇcnjaka u regularizaciju nije rezultirala uspješ- nijom predikcijom od metode regresijskih stabala niti za jedan pristup uvo ¯denja domenskog znanja struˇcnjaka. Rezultati pokazuju da razliˇciti naˇcini procjene penalizacijskih koeficijenta i sama struktura modela utjeˇcu na varijabilnost i pristranost modela. Ujedno, otvaraju daljnja pitanja u smislu kako posti´ci i da za kompleksnije skupove podataka domensko znanje struˇc- njaka doprinese smanjenju varijabilnosti i / ili pristranosti modela te poboljšanju uspješnosti predikcije. Less

Abstract (english)

This research aimed to develop a feature selection method for a linear regression model that in- corporates experts’ domain knowledge about the impact of input features on the output into the regularization process. Two methods of incorporating experts’ knowledge were distinguished: deterministic and metaheuristic. The proposed method was evaluated using data on energy costs and characteristics of public sector buildings in the Republic of Croatia, as well as three scena- rios of simulated... More data from the literature. Evaluation measures included prediction performance metrics (adjusted coefficient of determination, RMSE, MAE, and SMAPE) and, in the case of simulated data, the difference between actual and estimated regression coefficients. The met- hod was compared to regularization that does not include experts’ domain knowledge and to regression trees. The results on real data indicate that incorporating experts’ domain knowledge through a metaheuristic approach improved prediction performance compared to the mentioned methods, while incorporating domain knowledge through a deterministic approach improved prediction performance in certain cases. In the simulation experiment, introducing experts’ do- main knowledge improved some prediction performance measures over regularization without experts’ knowledge only with the metaheuristic approach, while the proposed method of in- troducing experts’ knowledge into regularization did not outperform the decision tree method in prediction for any approach of introducing experts’ knowledge. The results indicate that different ways of estimating penalization coefficients and the model structure itself affect the variability and bias of the model. Additionally, they raise further questions about how to ac- hieve improved prediction performance and reduced variability and / or bias by incorporating experts’ domain knowledge in more complex data sets. Less

• Keywords: odabir značajki; linearna regresija; regularizacija; domensko znanje stručnjaka; metaheuristika; pretraživanje kukavice; regresijska stabla; predikcija; energetski troškovi; zgrade javne namjene; simulacije
• Keywords (english): feature selection; linear regression; regularization; experts’ domain knowledge; metaheuristics; cuckoo search; regression trees; prediction; energy costs; public buildings; simulations
• Language: croatian
• URN:NBN: urn:nbn:hr:211:096890
• Promotion: 2025
• Study programme: Title: Postgraduate doctoral study in Information Science; Study programme type: university; Study level: postgraduate; Academic / professional title: doktor/doktorica znanosti, područje društvenih znanosti, polje informacijske i komunikacijske znanosti (doktor/doktorica znanosti, područje društvenih znanosti, polje informacijske i komunikacijske znanosti)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2025-03-05 19:09:42