The so-called linear least squares method can be understood as a continuation of solving equations. The difference is that when the unknown quantity is far smaller than the number of equations, an unsolvable problem will be obtained. The essence of the least squares method is to assign values to unknown numbers while ensuring the minimum error.
The least squares method is a very classic algorithm, and we have been exposed to this name in high school. It is an extremely commonly used algorithm. I have previously written about the principle of linear least squares and implemented it in Python: least squares and its Python implementation; and how to call nonlinear least squares in scipy: nonlinear least squares(Supplementary content at the end of the article);There is also the least squares method of sparse matrices: sparse matrix least squares method.
The following describes the linear least squares method implemented in numpy and scipy, and compares the speed of the two.
numpy implementation
The least squares method is implemented in numpy, that is, lstsq(a,b) is used to solve x similar to a@x=b, where a is M× N matrix; then when b is a vector of M rows, it is just equivalent to solving a system of linear equations. For a system of equations like Ax=b, if A is a full-rank simulation, it can be expressed as x=A−1b, otherwise it can be expressed as x=(ATA)−1ATb.
When b is a matrix of M×K, then for each column, a set of x will be calculated.
There are 4 return values, which are the x obtained by fitting, the fitting error, the rank of matrix a, and the single-valued form of matrix a.
import numpy as np
np.random.seed(42)
M = np.random.rand(4,4)
x = np.arange(4)
y = M@x
xhat = np.linalg.lstsq(M,y)
print(xhat[0])
#[0. 1. 2. 3.]
Copy after login
scipy package
scipy.linalg also provides the least squares function. The function name is also lstsq, and its parameter list is
where a, b is Ax= b. Both provide overridable switches. Setting them to True can save running time. In addition, the function also supports finiteness checking, which is an option that many functions in linalg have. Its return value is the same as the least squares function in numpy.
cond is a floating point parameter, indicating the singular value threshold. When the singular value is less than cond, it will be discarded.
lapack_driver is a string option, indicating which algorithm engine in LAPACK is selected, optionally 'gelsd', 'gelsy', 'gelss'.
Finally, make a speed comparison between the two sets of least squares functions
from timeit import timeit
N = 100
A = np.random.rand(N,N)
b = np.arange(N)
timeit(lambda:np.linalg.lstsq(A, b), number=10)
# 0.015487500000745058
timeit(lambda:sl.lstsq(A, b), number=10)
# 0.011151800004881807
Copy after login
This time, the two are not too far apart The difference is that even if the matrix dimension is enlarged to 500, the two are about the same.
N = 500
A = np.random.rand(N,N)
b = np.arange(N)
timeit(lambda:np.linalg.lstsq(A, b), number=10)
0.389679799991427
timeit(lambda:sl.lstsq(A, b), number=10)
0.35642060000100173
Copy after login
Supplement
Python calls the nonlinear least squares method
Introduction and constructor
In In scipy, the purpose of the nonlinear least squares method is to find a set of functions that minimize the sum of squares of the error function, which can be expressed as the following formula
where ρ represents the loss function , can be understood as a preprocessing of fi(x).
scipy.optimize encapsulates the nonlinear least squares function least_squares, which is defined as
Among them, func and x0 are required parameters, func is the function to be solved, and x0 is the function input The initial value of , there is no default value for these two parameters, they are parameters that must be entered.
bound is the solution interval, the default is (−∞,∞). When verbose is 1, there will be a termination output. When verbose is 2, more information during the operation will be printed. In addition, the following parameters are used to control the error, which is relatively simple.
import numpy as np
from scipy.optimize import least_squares
def test(xs):
_sum = 0.0
for i in range(len(xs)):
_sum = _sum + (1-np.cos((xs[i]*i)/5)*(i+1))
return _sum
x0 = np.random.rand(5)
ret = least_squares(test, x0)
msg = f"最小值" + ", ".join([f"{x:.4f}" for x in ret.x])
msg += f"\nf(x)={ret.fun[0]:.4f}"
print(msg)
'''
最小值0.9557, 0.5371, 1.5714, 1.6931, 5.2294
f(x)=0.0000
'''
Copy after login
The above is the detailed content of How to call and implement the least squares method in Python. For more information, please follow other related articles on the PHP Chinese website!
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
PHP and Python have their own advantages and disadvantages, and the choice depends on project needs and personal preferences. 1.PHP is suitable for rapid development and maintenance of large-scale web applications. 2. Python dominates the field of data science and machine learning.
Python and JavaScript have their own advantages and disadvantages in terms of community, libraries and resources. 1) The Python community is friendly and suitable for beginners, but the front-end development resources are not as rich as JavaScript. 2) Python is powerful in data science and machine learning libraries, while JavaScript is better in front-end development libraries and frameworks. 3) Both have rich learning resources, but Python is suitable for starting with official documents, while JavaScript is better with MDNWebDocs. The choice should be based on project needs and personal interests.
Docker uses Linux kernel features to provide an efficient and isolated application running environment. Its working principle is as follows: 1. The mirror is used as a read-only template, which contains everything you need to run the application; 2. The Union File System (UnionFS) stacks multiple file systems, only storing the differences, saving space and speeding up; 3. The daemon manages the mirrors and containers, and the client uses them for interaction; 4. Namespaces and cgroups implement container isolation and resource limitations; 5. Multiple network modes support container interconnection. Only by understanding these core concepts can you better utilize Docker.
Python excels in automation, scripting, and task management. 1) Automation: File backup is realized through standard libraries such as os and shutil. 2) Script writing: Use the psutil library to monitor system resources. 3) Task management: Use the schedule library to schedule tasks. Python's ease of use and rich library support makes it the preferred tool in these areas.
In VS Code, you can run the program in the terminal through the following steps: Prepare the code and open the integrated terminal to ensure that the code directory is consistent with the terminal working directory. Select the run command according to the programming language (such as Python's python your_file_name.py) to check whether it runs successfully and resolve errors. Use the debugger to improve debugging efficiency.
VS Code is the full name Visual Studio Code, which is a free and open source cross-platform code editor and development environment developed by Microsoft. It supports a wide range of programming languages and provides syntax highlighting, code automatic completion, code snippets and smart prompts to improve development efficiency. Through a rich extension ecosystem, users can add extensions to specific needs and languages, such as debuggers, code formatting tools, and Git integrations. VS Code also includes an intuitive debugger that helps quickly find and resolve bugs in your code.
VS Code can be used to write Python and provides many features that make it an ideal tool for developing Python applications. It allows users to: install Python extensions to get functions such as code completion, syntax highlighting, and debugging. Use the debugger to track code step by step, find and fix errors. Integrate Git for version control. Use code formatting tools to maintain code consistency. Use the Linting tool to spot potential problems ahead of time.
VS Code extensions pose malicious risks, such as hiding malicious code, exploiting vulnerabilities, and masturbating as legitimate extensions. Methods to identify malicious extensions include: checking publishers, reading comments, checking code, and installing with caution. Security measures also include: security awareness, good habits, regular updates and antivirus software.
Backend Development
