算法 - 关于Python嵌套循环代码优化

Question

用Python实现K-means算法时候，要计算随机两个数之间的欧氏距离，数据量为5000行，但计算的时间却有500多秒，不知道有什么能优化，求指教，代码如下循环 {代码...} ed_relate {代码...} 数据data {代码...}

怪我咯 · Answer

The solution has been found. The problem lies in the two floats used to calculate the Euclidean distance
sum += (float(dataX[a])-float(dataY[a])) ** 2sum += (float(dataX[a])-float(dataY[a])) ** 2
先将data全转成 int
data = [[int(x) for x in row] for row in data] First convert all data into int
data = [[int(x) for x in row] for row in data]

In calculation, the speed is increased by 10 times🎜

高洛峰 · Answer

def ed_relate(dataX, dataY):
    '''
    :param dataX:第一行
    :param dataY: 第二行
    :return: 之间的相似度
    '''
    if len(dataX) == len(dataY):
        relate = math.sqrt(sum(((float(x)-float(y)**2) for x, y in zip(dataX, dataY))))
        return relate
    else:
        print 'len is not equal'
        return 0

Not tested, it should improve some performance.

伊谢尔伦 · Answer

It seems that there is no big problem with your existing code. I repeated your calculation 1,000 times, which is roughly equivalent to 20,000 pieces of data, and the time is 4.4 seconds.
You'd better profile it to see where the problem is.

迷茫 · Answer

There seems to be no better way in terms of performance, but I think your double loop can be written more elegantly, such as:

def unique_pairs(n):
    """在range(n)范围内生成索引对, 其他地方需要类似迭代可以复用unique_pairs生成器"""
    for i in range(n):
        for j in range(i+1, n):
            yield i, j
            

for i, j in unique_pairs(len(data)):  # 计算任意两点距离和
    random_sum += ed_relate(data[i][2:], data[j][2:])

阿神 · Answer

Using numpy and pandas