Indices and tables¶
User-Facing Classes¶
BCP¶
This is the core module of the library, containing the primary interface over the functionality of bcp. Along with its dependencies on Connection and DataFile, it serves as the entry point to the library. Simply create a Connection, pass it in to the BCP object, and then use load() or dump() to read data into and out of a database. See the methods below for examples.
-
class
bcp.core.
BCP
(connection)[source]¶ This is the interface over dialect-specific classes that provides generic methods to load/dump data to/from a database.
Parameters: connection (Connection) – a Connection object that contains authorization details and database details Example:
import bcp conn = bcp.Connection('host', 'mssql', 'username', 'password') my_bcp = bcp.BCP(conn)
-
dump
(query, output_file)[source]¶ This method provides an interface to the lower level dialect-specific BCP dump classes.
Parameters: - query (
str
) – the query whose results should be saved off to a file - output_file (DataFile) – the file to which the data should be saved, if no file is provided, one will be created in the BCP_DATA_DIR
Example:
import bcp conn = bcp.Connection(host='HOST', driver='mssql', username='USER', password='PASSWORD') my_bcp = bcp.BCP(conn) file = bcp.DataFile(file_path='path/to/file.csv', delimiter=',') my_bcp.dump(query='select * from sys.tables', output_file=file)
- query (
-
load
(input_file, table)[source]¶ This method provides an interface to the lower level dialect-specific BCP load classes
Parameters: - input_file (DataFile) – the file to be loaded into the database
- table (
str
) – the table in which to land the data
Example:
import bcp conn = bcp.Connection(host='HOST', driver='mssql', username='USER', password='PASSWORD') my_bcp = bcp.BCP(conn) file = bcp.DataFile(file_path='path/to/file.csv', delimiter=',') my_bcp.load(input_file=file, table='table_name')
-
Connections¶
This module contains data structures required to connect to a database. While Auth can be instantiated on its own, you would generally create it at the same time as the connection. Since the Auth object will contain credentials, it’s recommended to set these through secure means, such as environmental variables, so as to not accidentally check in your credentials to your source control. Even your host name can be considered sensitive data, depending on your and your company’s policies.
Example:
import os
import bcp
host = os.environ['HOST']
username = os.environ['USERNAME']
password = os.environ['PASSWORD']
conn = bcp.Connection(host, 'mssql', username, password)
my_bcp = bcp.BCP(conn)
-
class
bcp.connections.
Auth
(username=None, password=None)[source]¶ This data structure collects the username and password as an authentication object.
Parameters: - username (
Optional
[str
]) – username for the authorization - password (
Optional
[str
]) – password for the authorization
-
type
¶ This property identifies the authorization type depending on the username/password provided. The two options for authorization are Trusted and Credential. A Trusted connection is created when no username and no password are provided. In this case, the local user’s credentials and authorization method are used. A Credential connection is created when both a username and a password are provided. If only one of username and password are provided, this raises an InvalidCredentialException.
Return type: str
Returns: the type of connection (‘Trusted’ or ‘Credential’)
- username (
-
class
bcp.connections.
Connection
(driver, host, port=None, username=None, password=None)[source]¶ This data structure describes a connection to be used to instantiate a BCP instance. A host and driver must be supplied. A username/password combination can also be supplied upon instantiation to automatically create an associated Auth object. Alternatively, this can be set as an attribute after instantiation. If the username/password are not provided, the connection will assume a Trusted authorization in the meantime.
Parameters: - driver (
str
) – the type of database (mssql, etc.) - host (
str
) – the host where the database exists - port (
Optional
[int
]) – the port for the database server - username (
Optional
[str
]) – the username for authentication - password (
Optional
[str
]) – the password for authentication
- driver (
Files¶
This module contains data structures required to create and access files. Users will generally only need to use DataFile directly. LogFile and ErrorFile are used indirectly by the BCP classes.
Example:
from bcp import DataFile
# create a csv to write out my data
my_file = DataFile(delimiter=',')
print(my_file.path) # %HOME%/bcp/data/<timestamp>.csv
-
class
bcp.files.
DataFile
(file_path=None, delimiter=None)[source]¶ This is a handle to a data file.
Parameters: - file_path (
Optional
[Path
]) – the path object to the file, if not provided, a default using the current timestamp will be created - delimiter (
Optional
[str
]) – the field delimiter for the data file
- file_path (
-
class
bcp.files.
ErrorFile
(file_path=None)[source]¶ This is a handle to an error file.
Parameters: file_path ( Optional
[Path
]) – the path object to the file, if not provided, a default using the current timestamp will be created
Dialect Support¶
-
class
bcp.dialects.base.
BCPDump
(connection, query, file)[source]¶ This is the abstract base class for all driver specific Dump implementations. It contains required methods and default values for all subclasses.
Parameters: - connection (Connection) – the Connection object that points to the database from which we want to export data
- query (
str
) – the query for the data to be exported - file (DataFile) – the file to write the to, if not provided, a default will be created in the BCP_DATA_DIR
-
class
bcp.dialects.base.
BCPLoad
(connection, file, table)[source]¶ This is the abstract base class for all dialect-specific Dump implementations. It contains required methods and default values for all subclasses.
Parameters: - connection (Connection) – the Connection object that points to the database from which we want to export data
- file (DataFile) – the file to be loaded into the database
- table (
str
) – the table in which to land the data
MSSQL¶
This module contains MS SQL Server specific logic that works with the BCP command line utility. None of these classes are relevant beyond the scope of this library, and should not be used outside of the library.
-
class
bcp.dialects.mssql.
MSSQLBCP
[source]¶ This mixin provides properties for MSSQL’s BCP. It serves as a mixin for BCPLoad and BCPDump subclasses below.
-
command
¶ This allows you to see the command that will be executed via bcp without executing the command, similar to how sqlalchemy can show you the generated query or execute the generated query. This makes debugging a little easier.
Return type: str
Returns: the command that will be passed into the bcp command line utility
-
config
¶ This method will generate a configuration string. It supports the delimiter option.
Return type: str
Returns: a BCP formatted configuration string
-
logging
¶ This method will generate a log file string that will write the log file to the BCP_LOGGING_DIR directory.
Return type: str
Returns: a BCP formatted log file string
-
-
class
bcp.dialects.mssql.
MSSQLDump
(connection, query, file, character_data=True)[source]¶ This class is the MS SQL Server implementation of the BCP Dump operation.
Parameters: - connection (Connection) – the Connection object that points to the database from which we want to export data
- query (
str
) – the query defining the data to be exported - file (DataFile) – the file to which the data will be written
-
command
¶ This method will build the command that will export data from the supplied table to the supplied file.
Return type: str
Returns: the command that will be passed into the BCP command line utility
-
class
bcp.dialects.mssql.
MSSQLLoad
(connection, file, table, batch_size=10000, character_data=True)[source]¶ This class is the MS SQL Server implementation of the BCP Load operation.
Parameters: - connection (Connection) – the (mssql) Connection object that points to the database from which we want to export data
- file (DataFile) – the file whose data should be imported into the target database
- table (
str
) – the into which the data will be written - batch_size (
int
) – the number of records to read in one commit, defaulted to 10,000 - character_data (
bool
) – allows BCP to use character data, defaulted to True
-
command
¶ This method will build the command that will import data from the supplied file to the supplied table.
Return type: str
Returns: the command that will be passed into the BCP command line utility
-
config
¶ This method will generate a configuration string. It supports the delimiter and batch size options.
Return type: str
Returns: a BCP formatted configuration string
-
error
¶ This method will generate an error file string that will write the error file to the BCP_DATA_DIR directory.
Return type: str
Returns: a BCP formatted error file string
Configuration¶
Config¶
This module creates directories (or returns existing directories) to store logs, data and other artifacts.
Note
This application defaults to creating a ‘bcp’ directory inside of the user’s home directory. This would be the value of %USERPROFILE% on Windows or %HOME% on linux. This can be overridden by setting a value for the environmental variable BCP_ROOT_DIR. The structure within this root directory will always be the same.