Backend Development
Python Tutorial
Usage examples of common file operation functions of shutil module in Python
Usage examples of common file operation functions of shutil module in Python
The os module provides new/deleted/viewed file attributes for directories or files, and also provides path operations for files and directories. For example: absolute path, parent directory... However, the operations of os files should also include operations such as moving, copying, packaging, compression, and decompression. These os modules are not provided.
The shutdown discussed in this article is a supplement to file operations in os. --Move, copy, pack, compress and decompress,
shutil function function:
1 shutil.copyfileobj(fsrc, fdst[, length=16*1024])
Copy the content of the file to another file, and you can copy the content of a specified size
Let’s take a look at its source code first.
def copyfileobj(fsrc, fdst, length=16*1024):
"""copy data from file-like object fsrc to file-like object fdst"""
while 1:
buf = fsrc.read(length)
if not buf:
break
fdst.write(buf)
Attention! Among them, fsrc and fdst are both file objects, and they need to be opened before copying can be performed
import shutil
f1=open('name','r')
f2=open('name_copy','w+')
shutil.copyfileobj(f1,f2,length=16*1024)
2 shutil.copyfile(src,dst)
Copy the file content. Do you find it troublesome to copy the above files? You also need to manually open the file using the open function, which is not needed here. In fact, copyfile calls copyfileobj
def copyfile(src, dst, *, follow_symlinks=True):
if _samefile(src, dst):
raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
for fn in [src, dst]:
try:
st = os.stat(fn)
except OSError:
# File most likely does not exist
pass
else:
# XXX What about other special files? (sockets, devices...)
if stat.S_ISFIFO(st.st_mode):
raise SpecialFileError("`%s` is a named pipe" % fn)
if not follow_symlinks and os.path.islink(src):
os.symlink(os.readlink(src), dst)
else:
with open(src, 'rb') as fsrc:
with open(dst, 'wb') as fdst:
copyfileobj(fsrc, fdst)
return dst
shutil.copyfile('name','name_copy_2')
#一句就可以实现复制文件内容
3 shutil.copymode(src,dst)
Copy permission only, no changes to file content, groups and users.
def copymode(src, dst, *, follow_symlinks=True):
if not follow_symlinks and os.path.islink(src) and os.path.islink(dst):
if hasattr(os, 'lchmod'):
stat_func, chmod_func = os.lstat, os.lchmod
else:
return
elif hasattr(os, 'chmod'):
stat_func, chmod_func = os.stat, os.chmod
else:
return
st = stat_func(src)
chmod_func(dst, stat.S_IMODE(st.st_mode))
Look at the permissions of the two files first
[root@slyoyo python_test]# ls -l total 4 -rw-r--r--. 1 root root 79 May 14 05:17 test1 -rwxr-xr-x. 1 root root 0 May 14 19:10 test2
Run command
>>> import shutil
>>> shutil.copymode('test1','test2')
View results
[root@slyoyo python_test]# ls -l total 4 -rw-r--r--. 1 root root 79 May 14 05:17 test1 -rw-r--r--. 1 root root 0 May 14 19:10 test2
When we replace the target file with a non-existent file, an error occurs
>>> shutil.copymode('test1','test3')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/python/lib/python3.4/shutil.py", line 132, in copymode
chmod_func(dst, stat.S_IMODE(st.st_mode))
FileNotFoundError: [Errno 2] No such file or directory: 'test233'
4 shutil.copystat(src,dst)
Copy all status information, including permissions, groups, users, time, etc.
def copystat(src, dst, *, follow_symlinks=True):
def _nop(*args, ns=None, follow_symlinks=None):
pass
# follow symlinks (aka don't not follow symlinks)
follow = follow_symlinks or not (os.path.islink(src) and os.path.islink(dst))
if follow:
# use the real function if it exists
def lookup(name):
return getattr(os, name, _nop)
else:
# use the real function only if it exists
# *and* it supports follow_symlinks
def lookup(name):
fn = getattr(os, name, _nop)
if fn in os.supports_follow_symlinks:
return fn
return _nop
st = lookup("stat")(src, follow_symlinks=follow)
mode = stat.S_IMODE(st.st_mode)
lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
follow_symlinks=follow)
try:
lookup("chmod")(dst, mode, follow_symlinks=follow)
except NotImplementedError:
# if we got a NotImplementedError, it's because
# * follow_symlinks=False,
# * lchown() is unavailable, and
# * either
# * fchownat() is unavailable or
# * fchownat() doesn't implement AT_SYMLINK_NOFOLLOW.
# (it returned ENOSUP.)
# therefore we're out of options--we simply cannot chown the
# symlink. give up, suppress the error.
# (which is what shutil always did in this circumstance.)
pass
if hasattr(st, 'st_flags'):
try:
lookup("chflags")(dst, st.st_flags, follow_symlinks=follow)
except OSError as why:
for err in 'EOPNOTSUPP', 'ENOTSUP':
if hasattr(errno, err) and why.errno == getattr(errno, err):
break
else:
raise
_copyxattr(src, dst, follow_symlinks=follow)
5 shutil.copy(src,dst)
Copy the content and permissions of the file, copyfile first and then copymode
def copy(src, dst, *, follow_symlinks=True):
if os.path.isdir(dst):
dst = os.path.join(dst, os.path.basename(src))
copyfile(src, dst, follow_symlinks=follow_symlinks)
copymode(src, dst, follow_symlinks=follow_symlinks)
return dst
6 shutil.copy2(src,dst)
Copy the contents of the file as well as all of the file's status information. Copyfile first and then copystat
def copy2(src, dst, *, follow_symlinks=True):
"""Copy data and all stat info ("cp -p src dst"). Return the file's
destination."
The destination may be a directory.
If follow_symlinks is false, symlinks won't be followed. This
resembles GNU's "cp -P src dst".
"""
if os.path.isdir(dst):
dst = os.path.join(dst, os.path.basename(src))
copyfile(src, dst, follow_symlinks=follow_symlinks)
copystat(src, dst, follow_symlinks=follow_symlinks)
return dst
7 shutil.copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2,ignore_dangling_symlinks=False)
Recursively copy file content and status information
def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2,
ignore_dangling_symlinks=False):
names = os.listdir(src)
if ignore is not None:
ignored_names = ignore(src, names)
else:
ignored_names = set()
os.makedirs(dst)
errors = []
for name in names:
if name in ignored_names:
continue
srcname = os.path.join(src, name)
dstname = os.path.join(dst, name)
try:
if os.path.islink(srcname):
linkto = os.readlink(srcname)
if symlinks:
# We can't just leave it to `copy_function` because legacy
# code with a custom `copy_function` may rely on copytree
# doing the right thing.
os.symlink(linkto, dstname)
copystat(srcname, dstname, follow_symlinks=not symlinks)
else:
# ignore dangling symlink if the flag is on
if not os.path.exists(linkto) and ignore_dangling_symlinks:
continue
# otherwise let the copy occurs. copy2 will raise an error
if os.path.isdir(srcname):
copytree(srcname, dstname, symlinks, ignore,
copy_function)
else:
copy_function(srcname, dstname)
elif os.path.isdir(srcname):
copytree(srcname, dstname, symlinks, ignore, copy_function)
else:
# Will raise a SpecialFileError for unsupported file types
copy_function(srcname, dstname)
# catch the Error from the recursive copytree so that we can
# continue with other files
except Error as err:
errors.extend(err.args[0])
except OSError as why:
errors.append((srcname, dstname, str(why)))
try:
copystat(src, dst)
except OSError as why:
# Copying file access times may fail on Windows
if getattr(why, 'winerror', None) is None:
errors.append((src, dst, str(why)))
if errors:
raise Error(errors)
return dst
# version vulnerable to race conditions
[root@slyoyo python_test]# tree copytree_test/
copytree_test/
└── test
├── test1
├── test2
└── hahaha
[root@slyoyo test]# ls -l
total 0
-rw-r--r--. 1 python python 0 May 14 19:36 hahaha
-rw-r--r--. 1 python python 0 May 14 19:36 test1
-rw-r--r--. 1 root root 0 May 14 19:36 test2
>>> shutil.copytree('copytree_test','copytree_copy')
'copytree_copy'
[root@slyoyo python_test]# ls -l
total 12
drwxr-xr-x. 3 root root 4096 May 14 19:36 copytree_copy
drwxr-xr-x. 3 root root 4096 May 14 19:36 copytree_test
-rw-r--r--. 1 python python 79 May 14 05:17 test1
-rw-r--r--. 1 root root 0 May 14 19:10 test2
[root@slyoyo python_test]# tree copytree_copy/
copytree_copy/
└── test
├── hahaha
├── test1
└── test2
8 shutil.rmtree(path, ignore_errors=False, onerror=None)
Delete files recursively
def rmtree(path, ignore_errors=False, onerror=None):
if ignore_errors:
def onerror(*args):
pass
elif onerror is None:
def onerror(*args):
raise
if _use_fd_functions:
# While the unsafe rmtree works fine on bytes, the fd based does not.
if isinstance(path, bytes):
path = os.fsdecode(path)
# Note: To guard against symlink races, we use the standard
# lstat()/open()/fstat() trick.
try:
orig_st = os.lstat(path)
except Exception:
onerror(os.lstat, path, sys.exc_info())
return
try:
fd = os.open(path, os.O_RDONLY)
except Exception:
onerror(os.lstat, path, sys.exc_info())
return
try:
if os.path.samestat(orig_st, os.fstat(fd)):
_rmtree_safe_fd(fd, path, onerror)
try:
os.rmdir(path)
except OSError:
onerror(os.rmdir, path, sys.exc_info())
else:
try:
# symlinks to directories are forbidden, see bug #1669
raise OSError("Cannot call rmtree on a symbolic link")
except OSError:
onerror(os.path.islink, path, sys.exc_info())
finally:
os.close(fd)
else:
return _rmtree_unsafe(path, onerror)
9 shutil.move(src, dst)
Move files recursively
def move(src, dst):
real_dst = dst
if os.path.isdir(dst):
if _samefile(src, dst):
# We might be on a case insensitive filesystem,
# perform the rename anyway.
os.rename(src, dst)
return
real_dst = os.path.join(dst, _basename(src))
if os.path.exists(real_dst):
raise Error("Destination path '%s' already exists" % real_dst)
try:
os.rename(src, real_dst)
except OSError:
if os.path.islink(src):
linkto = os.readlink(src)
os.symlink(linkto, real_dst)
os.unlink(src)
elif os.path.isdir(src):
if _destinsrc(src, dst):
raise Error("Cannot move a directory '%s' into itself '%s'." % (src, dst))
copytree(src, real_dst, symlinks=True)
rmtree(src)
else:
copy2(src, real_dst)
os.unlink(src)
return real_dst
10 make_archive(base_name, format, root_dir=None, base_dir=None, verbose=0,dry_run=0, owner=None, group=None, logger=None)
Compressed and packaged
def make_archive(base_name, format, root_dir=None, base_dir=None, verbose=0,
dry_run=0, owner=None, group=None, logger=None):
save_cwd = os.getcwd()
if root_dir is not None:
if logger is not None:
logger.debug("changing into '%s'", root_dir)
base_name = os.path.abspath(base_name)
if not dry_run:
os.chdir(root_dir)
if base_dir is None:
base_dir = os.curdir
kwargs = {'dry_run': dry_run, 'logger': logger}
try:
format_info = _ARCHIVE_FORMATS[format]
except KeyError:
raise ValueError("unknown archive format '%s'" % format)
func = format_info[0]
for arg, val in format_info[1]:
kwargs[arg] = val
if format != 'zip':
kwargs['owner'] = owner
kwargs['group'] = group
try:
filename = func(base_name, base_dir, **kwargs)
finally:
if root_dir is not None:
if logger is not None:
logger.debug("changing back to '%s'", save_cwd)
os.chdir(save_cwd)
return filename
base_name: The compressed file name or path name
format: Compression or packaging format "zip", "tar", "bztar" or "gztar"
root_dir: Which directory or file to package (that is, the source file)
>>> shutil.make_archive('tarball','gztar',root_dir='copytree_test')
[root@slyoyo python_test]# ls -l
total 12
drwxr-xr-x. 3 root root 4096 May 14 19:36 copytree_copy
drwxr-xr-x. 3 root root 4096 May 14 19:36 copytree_test
-rw-r--r--. 1 root root 0 May 14 21:12 tarball.tar.gz
-rw-r--r--. 1 python python 79 May 14 05:17 test1
-rw-r--r--. 1 root root 0 May 14 19:10 test2
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1378
52
The 2-Hour Python Plan: A Realistic Approach
Apr 11, 2025 am 12:04 AM
You can learn basic programming concepts and skills of Python within 2 hours. 1. Learn variables and data types, 2. Master control flow (conditional statements and loops), 3. Understand the definition and use of functions, 4. Quickly get started with Python programming through simple examples and code snippets.
How to read redis queue
Apr 10, 2025 pm 10:12 PM
To read a queue from Redis, you need to get the queue name, read the elements using the LPOP command, and process the empty queue. The specific steps are as follows: Get the queue name: name it with the prefix of "queue:" such as "queue:my-queue". Use the LPOP command: Eject the element from the head of the queue and return its value, such as LPOP queue:my-queue. Processing empty queues: If the queue is empty, LPOP returns nil, and you can check whether the queue exists before reading the element.
How to start the server with redis
Apr 10, 2025 pm 08:12 PM
The steps to start a Redis server include: Install Redis according to the operating system. Start the Redis service via redis-server (Linux/macOS) or redis-server.exe (Windows). Use the redis-cli ping (Linux/macOS) or redis-cli.exe ping (Windows) command to check the service status. Use a Redis client, such as redis-cli, Python, or Node.js, to access the server.
How to read data from redis
Apr 10, 2025 pm 07:30 PM
To read data from Redis, you can follow these steps: 1. Connect to the Redis server; 2. Use get(key) to get the value of the key; 3. If you need string values, decode the binary value; 4. Use exists(key) to check whether the key exists; 5. Use mget(keys) to get multiple values; 6. Use type(key) to get the data type; 7. Redis has other read commands, such as: getting all keys in a matching pattern, using cursors to iterate the keys, and sorting the key values.
Python vs. C : Applications and Use Cases Compared
Apr 12, 2025 am 12:01 AM
Python is suitable for data science, web development and automation tasks, while C is suitable for system programming, game development and embedded systems. Python is known for its simplicity and powerful ecosystem, while C is known for its high performance and underlying control capabilities.
What types of files are composed of oracle databases?
Apr 11, 2025 pm 03:03 PM
Oracle database file structure includes: data file: storing actual data. Control file: Record database structure information. Redo log files: record transaction operations to ensure data consistency. Parameter file: Contains database running parameters to optimize performance. Archive log file: Backup redo log file for disaster recovery.
How to find keys with redis
Apr 10, 2025 pm 05:45 PM
There are several ways to find keys in Redis: Use the SCAN command to iterate over all keys by pattern or condition. Use GUI tools such as Redis Explorer to visualize the database and filter keys by name or schema. Write external scripts to query keys using the Redis client library. Subscribe to keyspace notifications to receive alerts when key changes.
How to obtain redis login permission
Apr 10, 2025 pm 07:18 PM
To obtain Redis login permission, you need to perform the following steps: 1. Create a username and password; 2. Allow remote connections; 3. Restart the Redis server; 4. Connect using the Redis CLI or programming language.
