Python comes with many out of the box modules (like os, subprocess, and shutil) to support File I/O operations. And in this post, you’ll get to see some unique ways to copy a file in Python. So, let’s get started to learn the nine different methods to do Python copy file operation.
Before you begin, you must understand why is it so important to know which copy file method in Python is best for you. It is because file I/O operations are performance intensive and often lead to bottlenecks. That’s why you should opt for the best possible approach as per the design of your application.
Some programs which use shared resources would prefer to copy a file in blocking mode whereas some may want to do it asynchronously. For example – using a thread to copy a file or starting a separate process to do it. Another point to consider is the platform portability. It means you should know the target OS (Windows/Linux/Mac OS X etc.) wherever you’ll run the program.
With the shutil module, you can automate copying both the files and folders. This module follows an optimized design. It saves you from doing the time-intensive operations like the opening, reading, writing, and closing of a file when there is no real processing need. It is full of utility functions and methods which can let you do tasks like copying, moving or removing files and folders.
How to Do Python Copy File – 9 Ways to Learn.
Here are the nine methods to demonstrate “How to copy a file in Python?”.
- shutil copyfile() method
- shutil copy() method
- shutil copyfileobj() method
- shutil copy2() method
- os popen method
- os system() method
- threading Thread() method
- subprocess call() method
- subprocess check_output() method
This method copies the content of the source to the destination only if the target is writable. If you don’t have the right permissions, then it will raise an IOError.
It works by opening the input file for reading while ignoring its file type.
Next, it doesn’t treat special files any differently and won’t create their clones.
The copyfile() method makes use of lower-level function copyfileobj() underneath. It takes file names as arguments, opens them and passes file handles to copyfileobj(). There is one optional third argument in this method which you can use to specify the buffer length. It’ll then open the file for reading in chunks of the specified buffer size. However, the default behavior is to read the entire file in one go.
Following are the points to know about the copyfile() method.
- It copies the contents of the source to a file named as the destination.
- If the destination isn’t writable, then the copy operation would result in an IOError exception.
- It will return the SameFileError if both the source and destination files are the same.
- However, if the destination pre-exists with a different name, then the copy will overwrite its content.
- Error 13 will occur if the destination is a directory which means this method won’t copy to a folder.
- It doesn’t support copying files such as character or block devices and the pipes.
# Python Copy File - Sample Code from shutil import copyfile from sys import exit source = input("Enter source file with full path: ") target = input("Enter target file with full path: ") # adding exception handling try: copyfile(source, target) except IOError as e: print("Unable to copy file. %s" % e) exit(1) except: print("Unexpected error:", sys.exc_info()) exit(1) print("\nFile copy done!\n") while True: print("Do you like to print the file ? (y/n): ") check = input() if check == 'n': break elif check == 'y': file = open(target, "r") print("\nHere follows the file content:\n") print(file.read()) file.close() print() break else: continue
copyfile(source_file, [destination_file or dest_dir])
The copy() method functions like the “cp” command in Unix. It means if the target is a folder, then it’ll create a new file inside it with the same name (basename) as the source file. Also, this method will sync the permissions of the target file with the source after copying its content. It too throws the SameFileError if you are copying the same file.
import os import shutil source = 'current/test/test.py' target = '/prod/new' assert not os.path.isabs(source) target = os.path.join(target, os.path.dirname(source)) # create the folders if not already exists os.makedirs(target) # adding exception handling try: shutil.copy(source, target) except IOError as e: print("Unable to copy file. %s" % e) except: print("Unexpected error:", sys.exc_info())
copy() vs copyfile() –
- The copy() also sets the permission bits while copying the contents whereas the copyfile() only copies the data.
- The copy() will copy a file if the destination is a directory whereas the copyfile() will fail with error 13.
- Interestingly, the copyfile() method utilizes the copyfileobj() method in its implementation whereas the copy() method makes use of the copyfile() and copymode() functions in turn.
- Point-3 makes it apparent that copyfile() would be a bit faster than the copy() as the latter has an additional task (preserving the permissions) at hand.
This method copies the file to a target path or file object. If the target is a file object, then you need to close it explicitly after the calling the copyfileobj(). It assumes an optional argument (the buffer size) which you can use to supply the buffer length. It is the number of bytes kept in memory during the copy process. The default size that system use is 16KB.
from shutil import copyfileobj status = False if isinstance(target, string_types): target = open(target, 'wb') status = True try: copyfileobj(self.stream, target, buffer_size) finally: if status: target.close()
However, the copy2() method functions like the copy(). But it also gets the access and modification times added in the meta-data while copying the data. Copying the same file would result in SameFileError.
from shutil import * import os import time from os.path import basename def displayFileStats(filename): file_stats = os.stat(basename(filename)) print('\tMode :', file_stats.st_mode) print('\tCreated :', time.ctime(file_stats.st_ctime)) print('\tAccessed:', time.ctime(file_stats.st_atime)) print('\tModified:', time.ctime(file_stats.st_mtime)) os.mkdir('test') print('SOURCE:') displayFileStats(__file__) copy2(__file__, 'testfile') print('TARGET:') displayFileStats(os.path.realpath(os.getcwd() + './test/testfile'))
copy() vs copy2()
- The copy() only sets permission bits whereas copy2() also updates the file metadata with timestamps.
- The copy() method calls copyfile() and copymode() internally whereas copy2() replaces the call to copymode() with copystat().
copymode() vs copystat()
copymode(source, target, *, follow_symlinks=True)
- It intends to copy the permission bits from source to the target files.
- The file contents, owner, and group remain unchanged. The arguments passed are strings.
- If the follow_symlinks arg is false and the first two args are symlinks, then copymode() will try to update the target link, not the actual file it is pointing.
copystat(source, target, *, follow_symlinks=True)
- It attempts to preserve the permission bits, last used time/update time, and flags of the target file.
- The copystat() includes the extended attributes while copying on Linux. The file contents/owner/group remain unchanged.
- If the follow_symlinks arg is false and the first two args are symlinks, then copystat() will update them, not the files they point.
This method creates a pipe to or from the command. It returns an open file object which connects to a pipe. You can use it for reading or writing according to the file mode, i.e., ‘r’ (default) or ‘w.’
os.popen(command[, mode[, bufsize]])
- mode – It can be ‘r’ (default) or ‘w’.
- bufsize – If its value is 0, then no buffering will occur. If the bufsize is 1, then line buffering will take place while accessing the file. If you provide a value greater than 1, then buffering will occur with the specified buffer size. However, for a negative value, the system will assume the default buffer size.
For Windows OS.
import os os.popen('copy 1.txt.py 2.txt.py')
For Linux OS.
import os os.popen('cp 1.txt.py 2.txt.py')
The system() method allows you to instantly execute any OS command or a script in the subshell.
You need to pass the command or the script as an argument to the system() call. Internally, this method calls the standard C library function.
Its return value is the exit status of the command.
For Windows OS.
import os os.system('copy 1.txt.py 2.txt.py')
For Linux OS.
import os os.system('cp 1.txt.py 2.txt.py')
If you want to copy a file asynchronously, then use the below method. In this, we’ve used Python’s threading module to run the copy operation in the background.
While using this method, please make sure to employ locking to avoid deadlocks. You may face it if your application is using multiple threads reading/writing a file.
import shutil from threading import Thread src="1.txt.py" dst="3.txt.py" Thread(target=shutil.copy, args=[src, dst]).start()
The subprocess module gives a simple interface to work with child processes. It enables us to launch subprocesses, attach to their input/output/error pipes, and retrieve the return values.
The subprocess module aims to replace the legacy modules and functions like – os.system, os.spawn*, os.popen*, popen2.*.
It exposes a call() method to invoke system commands to execute user tasks.
import subprocess src="1.txt.py" dst="2.txt.py" cmd='copy "%s" "%s"' % (src, dst) status = subprocess.call(cmd, shell=True) if status != 0: if status < 0: print("Killed by signal", status) else: print("Command failed with return code - ", status) else: print('Execution of %s passed!\n' % cmd)
With subprocess’s check_output() method, you can run an external command or a program and capture its output. It also supports pipes.
import os, subprocess src=os.path.realpath(os.getcwd() + "./1.txt.py") dst=os.path.realpath(os.getcwd() + "./2.txt.py") cmd='copy "%s" "%s"' % (src, dst) status = subprocess.check_output(['copy', src, dst], shell=True) print("status: ", status.decode('utf-8'))
Summary – Python Copy File Methods to Learn
Even if you are a seasoned Python programmer, you might have liked the different methods to copy a file. Python is so rich in features that one can quickly utilize them to perform a single task in many ways.
The os, subprocess and the shutil modules in Python bear tremendous support for file I/O operations. However, here, we’ve only covered the Python file copy methods. But you’ll see a lot of new and intuitive ways to do other things as well in our future posts.