University of Twente Student Theses


Local Optimization Algorithm for minimizing the Least Squares Criterion over Lipschitz Functions

Waninge, L.N. (2023) Local Optimization Algorithm for minimizing the Least Squares Criterion over Lipschitz Functions.

[img] PDF
Abstract:We consider the basic regression model and the least squares estimator over the Lipschitz-1 functions. Two algorithms that approximate a function based on an i.i.d. sample are compared. The first algorithm utilizes a quadratic programming (QP) approach, where the objective function corresponds to the least squares estimator (LSE), and the constraints are determined by the Lipschitz condition. Beliakov's algorithm, which reduces the time complexity of the QP, is implemented and used as a benchmark. The second algorithm is based on a local optimality characterization for Lipschitz least squares estimators. We call this the Local Optimization algorithm. Both algorithms are tested for different data distributions, sample sizes, and test functions. The analysis presented in this article focuses on two fundamental metrics: the running time and the maximum error (supremum norm). We can conclude that the local optimization algorithm is able to output comparable results as the Beliakov algorithm in a fraction of the time (especially for large sample sizes). However, it is important to note that when dealing with non-uniform distributed data, especially where a small number of data points cover a larger portion of the entire interval, the algorithm needs to be carefully tuned to ensure comparable and reliable output.
Item Type:Essay (Bachelor)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:31 mathematics
Programme:Applied Mathematics BSc (56965)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page