Reading and writing interfaces for miniSEED to/from files or URLs. More...

Data Structures
struct	LMIO
	Type definition for data source I/O: file-system versus URL. More...

struct	MS3FileParam
	State container for reading miniSEED records from files or URLs. More...

Macros
#define	LMIO_INITIALIZER
	Initialializer for the internal stream handle LMIO.

#define	MS3FileParam_INITIALIZER
	Initialializer for the internal file or URL I/O parameters MS3FileParam.

Functions
int	ms3_readmsr (MS3Record *ppmsr, const char mspath, uint32_t flags, int8_t verbose)
	Read miniSEED records from a file or URL.

int	ms3_readmsr_r (MS3FileParam ppmsfp, MS3Record ppmsr, const char *mspath, uint32_t flags, int8_t verbose)
	Read miniSEED records from a file or URL in a thread-safe way.

int	ms3_readmsr_selection (MS3FileParam ppmsfp, MS3Record ppmsr, const char mspath, uint32_t flags, const MS3Selections selections, int8_t verbose)
	Read miniSEED records from a file or URL with optional selection.

int	ms3_readtracelist (MS3TraceList *ppmstl, const char mspath, const MS3Tolerance *tolerance, int8_t splitversion, uint32_t flags, int8_t verbose)
	Read miniSEED from a file into a trace list.

int	ms3_readtracelist_timewin (MS3TraceList *ppmstl, const char mspath, const MS3Tolerance *tolerance, nstime_t starttime, nstime_t endtime, int8_t splitversion, uint32_t flags, int8_t verbose)
	Read miniSEED from a file into a trace list, with time range selection.

int	ms3_readtracelist_selection (MS3TraceList *ppmstl, const char mspath, const MS3Tolerance tolerance, const MS3Selections selections, int8_t splitversion, uint32_t flags, int8_t verbose)
	Read miniSEED from a file into a trace list, with selection filtering.

int	ms3_url_useragent (const char program, const char version)
	Set User-Agent header for URL-based requests.

int	ms3_url_userpassword (const char *userpassword)
	Set authentication credentials for URL-based requests.

int	ms3_url_addheader (const char *header)
	Add header to any URL-based requests.

void	ms3_url_freeheaders (void)
	Free all set headers for URL-based requests.

int64_t	msr3_writemseed (MS3Record msr, const char mspath, int8_t overwrite, uint32_t flags, int8_t verbose)
	Write miniSEED from an MS3Record container to a file.

int64_t	mstl3_writemseed (MS3TraceList mst, const char mspath, int8_t overwrite, int maxreclen, int8_t encoding, uint32_t flags, int8_t verbose)
	Write miniSEED from an MS3TraceList container to a file.

int	libmseed_url_support (void)
	Run-time test for URL support in libmseed.

MS3FileParam *	ms3_mstl_init_fd (int fd)
	Initialize MS3FileParam parameters for a file descriptor.

Detailed Description

Reading and writing interfaces for miniSEED to/from files or URLs.

The miniSEED reading interfaces read from either regular files or URLs (if optional support is included). The miniSEED writing interfaces write to regular files.

URL support for reading is included by building the library with the LIBMSEED_URL variable defined, see the INSTALL instructions for more information. Only URL path-specified resources can be read, e.g. HTTP GET requests. More advanced POST or form-based requests are not supported.

The function libmseed_url_support() can be used as a run-time test to determine if URL support is included in the library.

Some parameters can be set that affect the reading of data from URLs, including:

set the User-Agent header with ms3_url_useragent()
set username and password for authentication with ms3_url_userpassword()
set arbitrary headers with ms3_url_addheader()
disable TLS/SSL peer and host verficiation by setting LIBMSEED_SSL_NOVERIFY environment variable

Diagnostics: Setting environment variable LIBMSEED_URL_DEBUG enables detailed verbosity of URL protocol exchanges.

See also: ms3_readmsr(); ms3_readmsr_selection(); ms3_readtracelist(); ms3_readtracelist_selection(); msr3_writemseed(); mstl3_writemseed()

Data Structure Documentation

◆ MS3FileParam

struct MS3FileParam

State container for reading miniSEED records from files or URLs.

In general these values should not be directly set or accessed. It is possible to allocate a structure and set the path, startoffset, and endoffset values for advanced usage. Note that file/URL start and end offsets can also be parsed from the path name as well.

Data Fields
char	path[512]	INPUT: File name or URL.
int64_t	startoffset	INPUT: Start position in input stream.
int64_t	endoffset	INPUT: End position in input stream, 0 == unknown (e.g. pipe)
int64_t	streampos	OUTPUT: Read position of input stream.
int64_t	recordcount	OUTPUT: Count of records read from this stream/file so far.
char *	readbuffer	INTERNAL: Read buffer, allocated internally.
int	readlength	INTERNAL: Length of data in read buffer.
int	readoffset	INTERNAL: Read offset in read buffer.
uint32_t	flags	INTERNAL: Stream reading state flags.
LMIO	input	INTERNAL: IO handle, file or URL.

Macro Definition Documentation

◆ LMIO_INITIALIZER

#define LMIO_INITIALIZER

Value:

  {                                                                        \
    .type = LMIO_NULL, .handle = NULL, .handle2 = NULL, .still_running = 0 \
  }

Initialializer for the internal stream handle LMIO.

◆ MS3FileParam_INITIALIZER

#define MS3FileParam_INITIALIZER

Value:

  {                                                               \
    .path = "", .startoffset = 0, .endoffset = 0, .streampos = 0, \
    .recordcount = 0, .readbuffer = NULL, .readlength = 0,        \
    .readoffset = 0, .flags = 0, .input = LMIO_INITIALIZER        \
  }

Initialializer for the internal file or URL I/O parameters MS3FileParam.

Function Documentation

◆ ms3_readmsr()

int ms3_readmsr	(	MS3Record **	ppmsr,
		const char *	mspath,
		uint32_t	flags,
		int8_t	verbose )

extern

Read miniSEED records from a file or URL.

This routine is a wrapper for ms3_readmsr_selection() that uses the global file reading parameters. This routine is not thread safe and cannot be used to read more than one stream at a time.

See ms3_readmsr_selection() for a further description of arguments.

Returns: Return value from ms3_readmsr_selection()

MessageOnError - this function logs a message on error

◆ ms3_readmsr_r()

int ms3_readmsr_r	(	MS3FileParam **	ppmsfp,
		MS3Record **	ppmsr,
		const char *	mspath,
		uint32_t	flags,
		int8_t	verbose )

extern

Read miniSEED records from a file or URL in a thread-safe way.

This routine is a wrapper for ms3_readmsr_selection() that uses the re-entrant capabilities. This routine is thread safe and can be used to read more than one stream at a time as long as separate MS3FileParam containers are used for each stream.

A MS3FileParam container will be allocated if *ppmsfp is NULL.

See ms3_readmsr_selection() for a further description of arguments.

Returns: Return value from ms3_readmsr_selection()

MessageOnError - this function logs a message on error

◆ ms3_readmsr_selection()

int ms3_readmsr_selection	(	MS3FileParam **	ppmsfp,
		MS3Record **	ppmsr,
		const char *	mspath,
		uint32_t	flags,
		const MS3Selections *	selections,
		int8_t	verbose )

extern

Read miniSEED records from a file or URL with optional selection.

This routine will open and read, with subsequent calls, all miniSEED records in specified stream (file or URL).

All stream reading parameters are stored in a MS3FileParam container and returned (via a pointer to a pointer) for the calling routine to use in subsequent calls. A MS3FileParam container will be allocated if *ppmsfp is NULL. This routine is thread safe and can be used to read multiple streams in parallel as long as the stream reading parameters are managed appropriately.

The flags argument are bit flags used to control the reading process. The following flags are supported:

MSF_SKIPNOTDATA - skip input that cannot be identified as miniSEED
MSF_UNPACKDATA data samples will be unpacked
MSF_VALIDATECRC Validate CRC (if present in format)
MSF_PNAMERANGE Parse byte range suffix from mspath

If MSF_PNAMERANGE is set in flags, the mspath will be searched for start and end byte offsets for the file or URL in the following format: 'PATH@START-END', where START and END are both optional and specified in bytes.

If selections is not NULL, the MS3Selections will be used to limit what is returned to the caller. Any data not matching the selections will be skipped.

After reading all the records in a stream the calling program should call this routine a final time with mspath set to NULL. This will close the input stream and free allocated memory.

Parameters

[out]	ppmsfp	Pointer-to-pointer of an MS3FileParam, which contains the state of stream reading across iterative calls of this function.
[out]	ppmsr	Pointer-to-pointer of an MS3Record, which will contain a parsed record on success.
[in]	mspath	File or URL to read
[in]	flags	Flags used to control parsing, see Control flags
[in]	selections	Specify limits to which data should be returned, see Data Selections
[in]	verbose	Controls verbosity, 0 means no diagnostic output

Returns: MS_NOERROR and populates an MS3Record struct, at *ppmsr, on successful read. On error, a (negative) libmseed error code is returned and *ppmsr is set to NULL.

Return values

MS_ENDOFFILE on reaching the end of a stream

See also: Data Selections

MessageOnError - this function logs a message on error

◆ ms3_readtracelist()

int ms3_readtracelist	(	MS3TraceList **	ppmstl,
		const char *	mspath,
		const MS3Tolerance *	tolerance,
		int8_t	splitversion,
		uint32_t	flags,
		int8_t	verbose )

extern

Read miniSEED from a file into a trace list.

This is a simple wrapper for ms3_readtracelist_selection() that uses no selections.

See ms3_readtracelist_selection() for a further description of arguments.

Returns: Return value from ms3_readtracelist_selection()

MessageOnError - this function logs a message on error

See also: Trace List

◆ ms3_readtracelist_timewin()

int ms3_readtracelist_timewin	(	MS3TraceList **	ppmstl,
		const char *	mspath,
		const MS3Tolerance *	tolerance,
		nstime_t	starttime,
		nstime_t	endtime,
		int8_t	splitversion,
		uint32_t	flags,
		int8_t	verbose )

extern

Read miniSEED from a file into a trace list, with time range selection.

This is a wrapper for ms3_readtraces_selection() that creates a simple selection for a specified time window.

See ms3_readtracelist_selection() for a further description of arguments.

Returns: Return value from ms3_readtracelist_selection()

MessageOnError - this function logs a message on error

See also: Trace List

◆ ms3_readtracelist_selection()

int ms3_readtracelist_selection	(	MS3TraceList **	ppmstl,
		const char *	mspath,
		const MS3Tolerance *	tolerance,
		const MS3Selections *	selections,
		int8_t	splitversion,
		uint32_t	flags,
		int8_t	verbose )

extern

Read miniSEED from a file into a trace list, with selection filtering.

This routine will open and read all miniSEED records in specified file and populate a MS3TraceList, allocating this struture if needed. This routine is thread safe.

If selections is not NULL, the MS3Selections will be used to limit which records are added to the trace list. Any data not matching the selections will be skipped.

As this routine reads miniSEED records it attempts to construct continuous time series, merging segments when possible. See mstl3_addmsr() for details of tolerance.

The splitversion flag controls whether data are grouped according to data publication version (or quality for miniSEED 2.x). See mstl3_addmsr() for full details.

If the MSF_RECORDLIST flag is set in flags, a MS3RecordList will be built for each MS3TraceSeg. The MS3RecordPtr entries contain the location of the data record, bit flags, extra headers, etc.

Parameters

[out]	ppmstl	Pointer-to-pointer to a MS3TraceList to populate
[in]	mspath	File to read
[in]	tolerance	Tolerance function pointers as MS3Tolerance
[in]	selections	Pointer to MS3Selections for limiting data
[in]	splitversion	Flag to control splitting of version/quality
[in]	flags	Flags to control reading, see ms3_readmsr_selection()
[in]	verbose	Controls verbosity, 0 means no diagnostic output

Returns: MS_NOERROR and populates an MS3TraceList struct at *ppmstl on success, otherwise returns a (negative) libmseed error code.

MessageOnError - this function logs a message on error

See also: Trace List; Data Selections

◆ ms3_url_useragent()

int ms3_url_useragent	(	const char *	program,
		const char *	version )

extern

Set User-Agent header for URL-based requests.

Configure global User-Agent header for URL-based requests generated by the library. The program and version values will be combined into the form "program/version" along with declarations of the library and URL-supporting dependency versions.

An error will be returned when the library was not compiled with URL support.

Parameters

[in]	program	Name of calling program
[in]	version	Version of calling program

Returns: 0 on succes and a negative library error code on error.

MessageOnError - this function logs a message on error

◆ ms3_url_userpassword()

int ms3_url_userpassword ( const char * userpassword )

extern

Set authentication credentials for URL-based requests.

Sets global user and password for authentication for URL-based requests generated by the library. The expected format of the credentials is: "[user name]:[password]" (without the square brackets).

An error will be returned when the library was not compiled with URL support.

Parameters

[in] userpassword User and password as user:password

Returns: 0 on succes and a negative library error code on error.

MessageOnError - this function logs a message on error

◆ ms3_url_addheader()

int ms3_url_addheader ( const char * header )

extern

Add header to any URL-based requests.

Sets global header to be included in URL-based requests generated by the library.

An error will be returned when the library was not compiled with URL support.

See also: ms3_url_freeheaders()

Parameters

[in] header Header in "key: value" format

Returns: 0 on succes and a negative library error code on error.

MessageOnError - this function logs a message on error

◆ ms3_url_freeheaders()

void ms3_url_freeheaders ( void )

extern

Free all set headers for URL-based requests.

Free all global headers for URL-based requests as set by ms3_url_addheader().

See also: ms3_url_addheader()

◆ msr3_writemseed()

int64_t msr3_writemseed	(	MS3Record *	msr,
		const char *	mspath,
		int8_t	overwrite,
		uint32_t	flags,
		int8_t	verbose )

extern

Write miniSEED from an MS3Record container to a file.

Pack MS3Record data into miniSEED record(s) by calling msr3_pack() and write to a specified file. The MS3Record container is used as a template for record(s) written to the file.

The overwrite flag controls whether a existing file is overwritten or not. If true (non-zero) any existing file will be replaced. If false (zero) new records will be appended to an existing file. In either case, new files will be created if they do not yet exist.

Parameters

[in,out]	msr	MS3Record containing data to write
[in]	mspath	File for output records
[in]	overwrite	Flag to control overwriting versus appending
[in]	flags	Flags controlling data packing, see msr3_pack()
[in]	verbose	Controls verbosity, 0 means no diagnostic output

Returns: the number of records written on success and -1 on error.

MessageOnError - this function logs a message on error

See also: msr3_pack()

◆ mstl3_writemseed()

int64_t mstl3_writemseed	(	MS3TraceList *	mstl,
		const char *	mspath,
		int8_t	overwrite,
		int	maxreclen,
		int8_t	encoding,
		uint32_t	flags,
		int8_t	verbose )

extern

Write miniSEED from an MS3TraceList container to a file.

Pack MS3TraceList data into miniSEED record(s) by calling mstl3_pack() and write to a specified file.

The overwrite flag controls whether a existing file is overwritten or not. If true (non-zero) any existing file will be replaced. If false (zero) new records will be appended to an existing file. In either case, new files will be created if they do not yet exist.

Parameters

[in,out]	mstl	MS3TraceList containing data to write
[in]	mspath	File for output records
[in]	overwrite	Flag to control overwriting versus appending
[in]	maxreclen	The maximum record length to create
[in]	encoding	encoding Encoding for data samples, see msr3_pack()
[in]	flags	Flags controlling data packing, see mstl3_pack() and msr3_pack()
[in]	verbose	Controls verbosity, 0 means no diagnostic output

Returns: the number of records written on success and -1 on error.

MessageOnError - this function logs a message on error

See also: mstl3_pack(); msr3_pack()

◆ libmseed_url_support()

int libmseed_url_support ( void )

extern

Run-time test for URL support in libmseed.

Returns: 0 when no URL suported is included, non-zero otherwise.

◆ ms3_mstl_init_fd()

MS3FileParam * ms3_mstl_init_fd ( int fd )

extern

Initialize MS3FileParam parameters for a file descriptor.

Initialize a MS3FileParam for reading from a specified fd (file descriptor).

The MS3FileParam should be used with ms3_readmsr_r() or ms3_readmsr_selection(). Once all data has been read from the stream, it will be closed during the cleanup call of those routines.

Parameters

[in] fd File descriptor for input reading

Returns: Allocated MS3FileParam on success and NULL on error.

MessageOnError - this function logs a message on error

Data Structures

Macros

Functions

Detailed Description

Data Structure Documentation

◆ MS3FileParam

Macro Definition Documentation

◆ LMIO_INITIALIZER

◆ MS3FileParam_INITIALIZER

Function Documentation

◆ ms3_readmsr()

◆ ms3_readmsr_r()

◆ ms3_readmsr_selection()

◆ ms3_readtracelist()

◆ ms3_readtracelist_timewin()

◆ ms3_readtracelist_selection()

◆ ms3_url_useragent()

◆ ms3_url_userpassword()

◆ ms3_url_addheader()

◆ ms3_url_freeheaders()

◆ msr3_writemseed()

◆ mstl3_writemseed()

◆ libmseed_url_support()

◆ ms3_mstl_init_fd()