Ingest files as lines between Hadoop clusters
The HDFS to HDFS Line Copy Application Template continuously ingests files as lines from between Hadoop clusters, retaining one-to-one file traceability.
Features of the enterprise-grade application template include these advantages:
- The application scales linearly with the number of record readers
- The application is fault-tolerant and can withstand node and cluster outages without data loss
- Highly performant, the application can perform as fast as the network allows
- DataTorrent’s template drastically simplifies custom logic, providing you business value with top connectivity and operational details of HDFS reader or writer
- Configuration is also simple: users need only provide source and destination HDFS paths
- Dramatic reduction in time-to-market and cost of operations
Download the application template and launch it to ingest data from one Hadoop cluster to another. Follow the tutorial videos or walkthrough document below to launch the template and add custom logic to process the data during ingestion.
Import, configure, and launch application template
Customize and Deploy
Add custom logic to the application template and go to production
Have feedback or want to learn more?