PHP-Hadoop-HDFS

Pure PHP unified wrapper for WebHDFS and CLI hadoop fs. Provides single interface for both, so you can decide which one to use depending on your tasks.

Requires PHP 5.3+

Documentation:

Each method has detailed docblocks, so using phpDocumentor should be enough to get started.

API:

• See docblocks for exceptions thrown by each method *

Instantiating CLI implementation:

$hdfs = new \Hdfs\Cli();

Instantiating WebHDFS implementation:

$hdfs = new \Hdfs\Web();
$hdfs->configure($host, $port, $user);

Change wrapper for local filesystem. Use it if you need hdfs to interact with another remote service instead of local FS

$hdfs->setFilesystemWrapper (\Hdfs\FilesystemWrapper $localfs)

Upload $localFile to Hadoop at $hdfsFile path

$hdfs->putFile ($localFile, $hdfsFile) ;

Download $hdfsFile from HDFS to $localFile in local filesystem

$hdfs->getFile ($hdfsFile, $localFile) ;

Remove file from HDFS

$hdfs->removeFile ($hdfsFile) ;

Set replication factor of a file $hdfsFile

$hdfs->setFileReplication ($hdfsFile, $factor) ;

Create $hdfsDir directory.

$hdfs->makeDir ($hdfsDir) ;

Remove $hdfsDir directory. Directory should be empty.

$hdfs->removeDir ($hdfsDir) ;

List contents of $hdfsDir directory. Returns array of \Hdfs\EntryStatus instances

$hdfs->readDir ($hdfsDir) ;

Get stats for $hdfsPath Returns \Hdfs\EntryStatus

$hdfs->stat ($hdfsPath) ;

Change permissions for $hdfsPath

$hdfs->changeMode ($hdfsPath, $mode) ;

Change owner and/or group for $hdfsPath. At least two arguments should be passed to this method. Either $owner or $group should be set.

$hdfs->changeOwner ($hdfsPath, $owner = null, $group = null) ;

Rename $hdfsSrcPath to $hdfsDstPath

$hdfs->rename ($hdfsSrcPath, $hdfsDstPath) ;

Check if $hdfsPath exists

$hdfs->isExists ($hdfsPath)

Check if $hdfsPath is directory

$hdfs->isDir ($hdfsPath)

Check if $hdfsPath is file

$hdfs->isFile ($hdfsPath)

Create directories recursively. If directory already exists and $mode/$owner/$group provided, then method just tries to apply those arguments to the last component in $hdfsDir.

$hdfs->makeDirRecursive ($hdfsDir, $mode = null, $owner = null, $group = null)

Change owner/group of $hdfsPath and all files&directories inside of $hdfsPath.

$hdfs->changeOwnerRecursive ($hdfsPath, $owner = null, $group = null)

Change permissions of $hdfsPath and all files&directories inside of $hdfsPath.

$hdfs->changeModeRecursive ($hdfsPath, $mode)

Uploads $localDir to $hdfsDir. If $hdfsDir does not exist, creates it. But if parent directory of $hdfsDir does not exist, throws exception.

$hdfs->putDir ($localDir, $hdfsDir, $isOverwrite = false)

Downloads $hdfsDir to $localDir. If $localDir does not exist, creates it. But if parent directory of $localDir does not exist, throws exception.

$hdfs->getDir ($hdfsDir, $localDir, $isOverwrite = false)

Copy file from one HDFS location to another HDFS location.

$hdfs->copyFile ($hdfsSrcPath, $hdfsDstPath, $isOverwrite = false)

Copy all files from one HDFS location to another HDFS location.

$hdfs->copyDir ($hdfsSrcPath, $hdfsDstPath, $isOverwrite = false)

Delete file or directory (not recursive).

$hdfs->delete ($hdfsPath)

Delete file or directory recursively.

$hdfs->deleteRecursive ($hdfsPath)

Get filesystem object size in bytes. Always returns 0 for directories.

$hdfs->getSize ($hdfsPath)

Get filesystem object size in bytes. Recursively gathers size information for directories.

$hdfs->getSizeRecursive ($hdfsPath)

Get file content without saving it to local filesystem.

$content = $hdfs->readFile ($hdfsFile, $offset = 0, $length = 0);

Write content to file.

$hdfs->writeFile ($hdfsFile, $content)