Do the amax and max functions in the cupy library in python go wrong when working with matrices with only one column or only one row?

问题内容

我尝试使用 cupy 进行 gpu 加速来实现用于机器学习和图像分类的 softmax 激活函数。我观察到，对于形状为 nx1 或 1xn 的数组，cupys max 函数会输出错误。然而，对于 nxa 的所有其他情况（其中 n 和 a 都是 1 以外的整数），它工作得很好。

我的代码：

def softmax_(z):
    max_z = cp.max(z, axis=0, keepdims=true)  # problematic max function
    exp_z = cp.exp(z - max_z)  # subtracting the maximum value for numerical stability
    sum_exp_z = cp.sum(exp_z, axis=0, keepdims=true)  # summing up the values
    return exp_z / sum_exp_z  # dividing them to get the softmax

array1 = cp.random.randn(3, 4)  # 3x4
array2 = cp.random.randn(5, 1)  # 5x1

print(softmax_(array1))  # no error
print(softmax_(array2))  # produces an error

我的操作系统错误，我对此缺乏经验：

oserror: [winerror 123] the filename, directory name, or volume label syntax is incorrect: 'c:\\users\\confidential\\.cupy\\jitify_cache\\tmp1pxgjv_g' -> 'c:\\users\\confidential/.cupy/jitify_cache/jitify_<unknown>_200200_12030_2_b3452ffa79e273adadd0403b6b0c05b78158b1e0.json'

数组 1 的输出

output:  [[0.17813469 0.20912114 0.19734889 0.30515635]  [0.42569072
0.47354802 0.4463671  0.20997539]  [0.39617459 0.31733085 0.356284   0.48486825]]

数组2的错误：

../../util_ptx.cuh(38): warning: util_type.cuh: [jitify] File not found 
../../util_ptx.cuh(41): warning: util_debug.cuh: [jitify] File not found
../../thread/thread_load.cuh(40): warning: ../util_ptx.cuh: [jitify] File not found
Traceback (most recent call last):
  File "c:\Users\confidential\Desktop\Projekte\Neural_network2\test.py", line 14, in <module>        
    print(softmax_(array2))
          ^^^^^^^^^^^^^^^^
  File "c:\Users\confidential\Desktop\Projekte\Neural_network2\test.py", line 4, in softmax_
    `max_Z = cp.max(Z, axis=0, keepdims=True)`
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\confidential\PycharmProjects\nunpy\venv\Lib\site-packages\cupy\_statistics\order.py", line 81, in amax
    return a.max(axis=axis, out=out, keepdims=keepdims)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cupy\_core\core.pyx", line 990, in cupy._core.core._ndarray_base.max
  File "cupy\_core\core.pyx", line 998, in cupy._core.core._ndarray_base.max
  File "cupy\_core\_routines_statistics.pyx", line 43, in cupy._core._routines_statistics._ndarray_max
  File "cupy\_core\_reduction.pyx", line 618, in cupy._core._reduction._SimpleReductionKernel.__call__
  File "cupy\_core\_reduction.pyx", line 370, in cupy._core._reduction._AbstractReductionKernel._call
  File "cupy\_core\_cub_reduction.pyx", line 689, in cupy._core._cub_reduction._try_to_call_cub_reduction
  File "cupy\_core\_cub_reduction.pyx", line 540, in cupy._core._cub_reduction._launch_cub    
  File "cupy\_util.pyx", line 64, in cupy._util.memoize.decorator.ret
  File "cupy\_core\_cub_reduction.pyx", line 240, in cupy._core._cub_reduction._SimpleCubReductionKernel_get_cached_function
  File "cupy\_core\_cub_reduction.pyx", line 223, in cupy._core._cub_reduction._create_cub_reduction_function
  File "cupy\_core\core.pyx", line 2254, in cupy._core.core.compile_with_cache
  File "C:\Users\confidential\PycharmProjects\nunpy\venv\Lib\site-packages\cupy\cuda\compiler.py", line 484, in _compile_module_with_cache
    return _compile_with_cache_cuda(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\confidential\PycharmProjects\nunpy\venv\Lib\site-packages\cupy\cuda\compiler.py", line 562, in _compile_with_cache_cuda
    ptx, mapping = compile_using_nvrtc(
                   ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\confidential\PycharmProjects\nunpy\venv\Lib\site-packages\cupy\cuda\compiler.py", line 319, in compile_using_nvrtc
    return _compile(source, options, cu_path,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\confidential\PycharmProjects\nunpy\venv\Lib\site-packages\cupy\cuda\compiler.py", line 284, in _compile
    options, headers, include_names = _jitify_prep(
                                      ^^^^^^^^^^^^^
  File "C:\Users\confidential\PycharmProjects\nunpy\venv\Lib\site-packages\cupy\cuda\compiler.py", line 233, in _jitify_prep
    jitify._init_module()
  File "cupy\cuda\jitify.pyx", line 212, in cupy.cuda.jitify._init_module
  File "cupy\cuda\jitify.pyx", line 233, in cupy.cuda.jitify._init_module
  File "cupy\cuda\jitify.pyx", line 209, in cupy.cuda.jitify._init_cupy_headers
  File "cupy\cuda\jitify.pyx", line 198, in cupy.cuda.jitify._init_cupy_headers_from_scratch  
  File "cupy\cuda\jitify.pyx", line 128, in cupy.cuda.jitify.dump_cache
OSError: [WinError 123] The syntax for the file name, directory name, or volume label is incorrect: 'C:\\Users\\confidential\\.cupy\\jitify_cache\\tmps16uxq46' -> 'C:\\Users\\confidential/.cupy/jitify_cache/jitify_<unknown>_200200_12030_2_b3452ffa79e273adadd0403b6b0c05b78158b1e0.json'

正确答案

您需要遵循的一些调试步骤。

1）更新cupy

pip install cupy --upgrade

2) 检查权限。 确保运行脚本的用户具有读取和写入 cupy_cache_dir 环境变量中指定的缓存目录的必要权限。

重塑输入数组 如果问题仍然存在，您可以尝试将输入数组重塑为 '(n,)' 的形状，而不是 '(n, 1)' 或 '(1, n)'。

4)禁用jit编译 您可以尝试通过将 cupy_cache_dir 环境变量设置为有效目录来禁用 jit 编译。

import cupy as cp
import os

os.environ['CUPY_CACHE_DIR'] = '/path/to/valid/directory'

将“/path/to/valid/directory”替换为 cupy 可以成功缓存已编译内核的目录。这可能会帮助您避免 oserror。

The above is the detailed content of Do the amax and max functions in the cupy library in python go wrong when working with matrices with only one column or only one row?. For more information, please follow other related articles on the PHP Chinese website!