kvikio::WebHdfsEndpoint Class Reference

A remote endpoint for Apache Hadoop WebHDFS. More...

#include <hdfs.hpp>

Inheritance diagram for kvikio::WebHdfsEndpoint:

Public Member Functions
	WebHdfsEndpoint (std::string url)
	Create an WebHDFS endpoint from a url. More...
	WebHdfsEndpoint (std::string host, std::string port, std::string remote_file_path, std::optional< std::string > username=std::nullopt)
	Create an WebHDFS endpoint from the host, port, file path and optionally username. More...
void	setopt (CurlHandle &curl) override
	Set needed connection options on a curl handle. More...
std::string	str () const override
	Get a description of this remote point instance. More...
std::size_t	get_file_size () override
	Get the size of the remote file. More...
void	setup_range_request (CurlHandle &curl, std::size_t file_offset, std::size_t size) override
	Set up the range request in order to read part of a file given the file offset and read size.
Public Member Functions inherited from kvikio::RemoteEndpoint
RemoteEndpointType	remote_endpoint_type () const noexcept
	Get the type of the remote file. More...

Static Public Member Functions
static bool	is_url_valid (std::string const &url) noexcept
	Whether the given URL is valid for the WebHDFS endpoints. More...

Additional Inherited Members
Protected Member Functions inherited from kvikio::RemoteEndpoint
	RemoteEndpoint (RemoteEndpointType remote_endpoint_type)
Protected Attributes inherited from kvikio::RemoteEndpoint
RemoteEndpointType	_remote_endpoint_type {RemoteEndpointType::AUTO}

Detailed Description

A remote endpoint for Apache Hadoop WebHDFS.

If KvikIO is run within a Docker, the argument --network host needs to be passed to the docker run command.

Definition at line 30 of file hdfs.hpp.

Constructor & Destructor Documentation

WebHdfsEndpoint() [1/2]

kvikio::WebHdfsEndpoint::WebHdfsEndpoint ( std::string  url )

explicit

Create an WebHDFS endpoint from a url.

Parameters

url	The WebHDFS HTTP/HTTPS url to the remote file.

WebHdfsEndpoint() [2/2]

kvikio::WebHdfsEndpoint::WebHdfsEndpoint ( std::string  host, std::string  port, std::string  remote_file_path, std::optional< std::string >  username = std::nullopt  )

explicit

Create an WebHDFS endpoint from the host, port, file path and optionally username.

Parameters

host	Host
port	Port
remote_file_path	Remote file path
username	User name

Member Function Documentation

get_file_size()

std::size_t kvikio::WebHdfsEndpoint::get_file_size ( )

overridevirtual

Get the size of the remote file.

Returns

The file size

Implements kvikio::RemoteEndpoint.

is_url_valid()

static bool kvikio::WebHdfsEndpoint::is_url_valid ( std::string const &  url )

staticnoexcept

Whether the given URL is valid for the WebHDFS endpoints.

Parameters

url

A URL.

Returns

Boolean answer.

setopt()

void kvikio::WebHdfsEndpoint::setopt ( CurlHandle &  curl )

overridevirtual

Set needed connection options on a curl handle.

Subsequently, a call to curl.perform() should connect to the endpoint.

Parameters

curl	The curl handle.

Implements kvikio::RemoteEndpoint.

str()

std::string kvikio::WebHdfsEndpoint::str ( ) const

overridevirtual

Get a description of this remote point instance.

Returns

A string description.

Implements kvikio::RemoteEndpoint.

---

The documentation for this class was generated from the following file:

• hdfs.hpp