HDFS to HDFS Line Copy

Download, configure, and deploy

Ingest files as lines between Hadoop clusters

The HDFS to HDFS Line Copy Application Template continuously ingests files as lines from between Hadoop clusters, retaining one-to-one file traceability.

Features of the enterprise-grade application template include these advantages:

  • The application scales linearly with the number of record readers
  • The application is fault-tolerant and can withstand node and cluster outages without data loss
  • Highly performant, the application can perform as fast as the network allows
  • DataTorrent’s template drastically simplifies custom logic, providing you business value with top connectivity and operational details of HDFS reader or writer
  • Configuration is also simple: users need only provide source and destination HDFS paths
  • Dramatic reduction in time-to-market and cost of operations

 

Download the application template and launch it to ingest data from one Hadoop cluster to another. Follow the tutorial videos or walkthrough document below to launch the template and add custom logic to process the data during ingestion.

Get Started

Import, configure, and launch application template

Customize and Deploy

Add custom logic to the application template and go to production