hdfs command refers to the command of Hadoop hdfs system. Its common commands include: 1. ls command; 2. cat command; 3. mkdir command; 4. rm command; 5. put command; 6. cp command ;7. copyFromLocal command; 8. get command; 9. copyToLocal command; 10. mv command, etc.
#The operating environment of this tutorial: linux5.9.8 system, Dell G3 computer.
What is the hdfs command?
refers to the command of Hadoop hdfs system.
To operate the hdfs system, you can use hadoop fs or hdfs dfs, both of which have the same effect. (The hadoop dfs command is no longer recommended)
Some commonly used commands in the Hadoop hdfs system
1. hadoop fs (hdfs dfs) file operations
ls displays all files or folders in the directory
Usage: hadoop fs -ls [uri form directory]
Example: hadoop fs –ls / Display all files and directories in the root directory
You can add the -R option to display all files in the directory
Example: hadoop fs -ls -R /
cat View file content
Usage: hadoop fs -cat URI [URI …]
Example: hadoop fs -cat /in/ test2.txt
mkdir Create directory
Usage: hadoop fs -mkdir [uri form directory]
Example: hadoop fs –mkdir /test
Create multi-level directories plus –p
Example: hadoop fs –mkdir -p /a/b/c
rm Delete a directory or file
Usage: hadoop fs -rm [file path] Delete a folder and add -r
Example: hadoop fs -rm /test1 .txt
Delete the folder and add -r,
Example: hadoop fs -rm -r /test
put Copy the file
Copy the file to the hdfs system, or read the file from the standard input. At this time, dst It is a file
Usage: hadoop fs -put
Example:
Hadoop fs -put /usr/wisedu/ temp/test1.txt /
Read files from standard input: hadoop fs -put -/in/myword
cp Copy files in the system
Usage: hadoopfs -cp URI [URI …]
Copy the file from the source path to the destination path. This command allows multiple source paths, in which case the target path must be a directory.
Example:
hadoop fs -cp /in/myword/word
copyFromLocal Copy local files to hdfs
Usage: hadoop fs -copyFromLocal
Similar to the put command except that the source path is limited to a local file
get copies the file to the local system
Usage: hadoop fs -get[-ignorecrc] [-crc]
Copy files to the local file system. Files that failed the CRC check can be copied using the -ignorecrc option. Use the -crc option to copy the file along with the CRC information.
Example: hadoop fs -get/word /usr/wisedu/temp/word.txt
copyToLocal Copy files to the local system
Usage: hadoop fs-copyToLocal [-ignorecrc] [-crc] URI
Similar to the get command except that the target path is limited to a local file.
Example: hadoop fs - copyToLocal/word /usr/wisedu/temp/word.txt
mv
will Files are moved from source path to destination path. This command allows multiple source paths, in which case the target path must be a directory. Moving files between different file systems is not allowed.
Usage: hadoop fs -mv URI [URI …]
Example: hadoop fs -mv /in/test2.txt /test2.txt
du Display file size
Display the size of all files in the directory.
Usage: hadoop fs -du URI [URI ...]
Example: hadoop fs -du /
To display the size of the current directory or folder, you can add the option -s
Example: hadoop fs -du -s /
touchz Create an empty file
Usage: hadoop fs -touchz URI [URI …]
Create an empty file with 0 bytes
Example: hadoop fs -touchz /empty.txt
chmod changes file permissions
Usage: hadoop fs -chmod[-R]
is similar to the chmod command on the Linux platform, changing the permissions of the file. Using -R will cause changes to be made recursively through the directory structure. The user of the command must be the owner of the file or the superuser.
Example: First create a normal user test: sudo useradd -m test
Then use wisedu user to create hello.txt file in hdfs system directory /a. At this time, test has the ability to read /a/hello The permissions of the .txt file are as shown below:
After switching back to the wisedu user, modify the permissions of the file to make the files in the /a directory unreadable to other users. Command: hadoop fs - chmod -R o-r /a As shown in the figure below, when switching back to the test user to view the /a/hello.txt file, it prompts that there is no permission:
chown Change the file Owner
Usage: hadoop fs -chown [-R] [OWNER][:[GROUP]] URI [URI]
Change the owner of the file. Using -R will cause changes to be made recursively through the directory structure. The user of the command must be a superuser.
Example: hadoop fs -chown -R test /a As shown below:
chgrp changes the group where the file is located
Usage: hadoop fs -chgrp [-R] GROUP URI [URI ...]
Change the group to which the file belongs. Using -R will cause changes to be made recursively through the directory structure. The user of the command must be the owner of the file or the superuser.
Example: hadoop fs -chgrp -R test /a As shown below:
2. hdfs dfsadmin management command
1) -report
View basic information and statistical information of the file system.
Example: hdfs dfsadmin -report
2) -safemode
enter | leave | get | wait: Safe mode command. Safe mode is a state of the NameNode in which the NameNode does not accept changes to the namespace (read-only); blocks are not copied or deleted. NameNode automatically enters safe mode when it starts. When the minimum percentage of configuration blocks meets the minimum number of replicas, it will automatically leave safe mode. Enter means to enter and leave means to leave.
Example: hdfs dfsadmin -safemode get
hdfsdfsadmin -safemode enter
3) -refreshNodes
Reread hosts and exclude files to make new Nodes or nodes that need to exit the cluster can be re-identified by the NameNode. This command is used when adding a node or deregistering a node.
Example: hdfs dfsadmin -refreshNodes
4) -finalizeUpgrade
End the HDFS upgrade operation. The DataNode deletes the previous version's working directory, and the NameNode does the same thereafter.
5) -upgradeProgress
status | details | force: Request the upgrade status of the current system | Details of the upgrade status | Forced upgrade operation
6) -metasave filename
Save the main data structure of NameNode to the
7) -setQuota
Set quota
8) -clrQuota
Clear quota settings for each directory
9) -help
Display help information
For more related knowledge, please visit the FAQ column!
The above is the detailed content of what is hdfs command. For more information, please follow other related articles on the PHP Chinese website!