OPAE C API Reference

The reference documentation for the OPAE C API is grouped into the following sections:

Types

The OPAE C API defines a number of types; most prominent are the types fpga_token, fpga_handle, and fpga_properties. All regular types are defined in [types.h](#types-h), while the values of enumeration types are defined in [types_enum.h](#types-enum-h).

types.h

Type definitions for FPGA API.

OPAE uses the three opaque types fpga_properties, fpga_token, and fpga_handle to create a hierarchy of objects that can be used to enumerate, reference, acquire, and query FPGA resources. This object model is designed to be extensible to account for different FPGA architectures and platforms.

Initialization

OPAEs management of the opaque types fpga_properties, fpga_token, and fpga_handle relies on the proper initialization of variables of these types. In other words, before doing anything with a variable of one of these opaque types, you need to first initialize them.

The respective functions that initizalize opaque types are:

This should intuitively make sense - fpgaGetProperties() creates fpga_properties objects, fpgaEnumerate() creates fpga_token objects, fpgaOpen() creates fpga_handle objects, and fpgaCloneProperties() and fpgaCloneToken() clone (create) fpga_properties and fpga_token objects, respectively.

Since these opaque types are interpreted as pointers (they are typedef’d to a void *), passing an uninitialized opaque type into any function except the respective initailzation function will result in undefined behaviour, because OPAE will try to follow an invalid pointer. Undefined behaviour in this case may include an unexpected error code, or an application crash.

Defines

FPGA_ERROR_NAME_MAX

Information about an error register

This data structure captures information about an error register exposed by an accelerator resource. The error API provides functions to retrieve these information structures from a particular resource.

FPGA_METRIC_STR_SIZE

FPGA Metric string size

Typedefs

typedef void *fpga_properties

Object for expressing FPGA resource properties

fpga_properties objects encapsulate all enumerable information about an FPGA resources. They can be used for two purposes: selective enumeration (discovery) and querying information about existing resources.

For selective enumeration, usually an empty fpga_properties object is created (using fpgaGetProperties()) and then populated with the desired criteria for enumeration. An array of fpga_properties can then be passed to fpgaEnumerate(), which will return a list of fpga_token objects matching these criteria.

For querying properties of existing FPGA resources, fpgaGetProperties() can also take an fpga_token and will return an fpga_properties object populated with information about the resource referenced by that token.

After use, fpga_properties objects should be destroyed using fpga_destroyProperties() to free backing memory used by the fpga_properties object.

typedef void *fpga_token

Token for referencing FPGA resources

An fpga_token serves as a reference to a specific FPGA resource present in the system. Holding an fpga_token does not constitute ownership of the FPGA resource - it merely allows the user to query further information about a resource, or to use fpgaOpen() to acquire ownership.

fpga_tokens are usually returned by fpgaEnumerate() or fpgaPropertiesGetParent(), and used by fpgaOpen() to acquire ownership and yield a handle to the resource. Some API calls also take fpga_tokens as arguments if they don’t require ownership of the resource in question.

typedef void *fpga_handle

Handle to an FPGA resource

A valid fpga_handle object, as populated by fpgaOpen(), denotes ownership of an FPGA resource. Note that ownership can be exclusive or shared, depending on the flags used in fpgaOpen(). Ownership can be released by calling fpgaClose(), which will render the underlying handle invalid.

Many OPAE C API functions require a valid token (which is synonymous with ownership of the resource).

typedef uint8_t fpga_guid[16]

Globally unique identifier (GUID)

GUIDs are used widely within OPAE for helping identify FPGA resources. For example, every FPGA resource has a guid property, which can be (and in the case of FPGA_ACCELERATOR resource primarily is) used for enumerating a resource of a specific type.

fpga_guid is compatible with libuuid’s uuid_t, so users can use libuuid functions like uuid_parse() to create and work with GUIDs.

typedef void *fpga_event_handle

Handle to an event object

OPAE provides an interface to asynchronous events that can be generated by different FPGA resources. The event API provides functions to register for these events; associated with every event a process has registered for is an fpga_event_handle, which encapsulates the OS-specific data structure for event objects.

After use, fpga_event_handle objects should be destroyed using fpgaDestroyEventHandle() to free backing memory used by the fpga_event_handle object.

typedef void *fpga_object

Object pertaining to an FPGA resource as identified by a unique name

An fpga_object represents either a device attribute or a container of attributes. Similar to filesystems, a ‘/’ may be used to seperate objects in an object hierarchy. Once on object is acquired, it may be used to read or write data in a resource attribute or to query sub-objects if the object is a container object. The data in an object is buffered and will be kept around until the object is destroyed. Additionally, the data in an attribute can by synchronized from the owning resource using the FPGA_OBJECT_SYNC flag during read operations. The name identifying the object is unique with respect to the resource that owns it. A parent resource may be identified by an fpga_token object, by an fpga_handle object, or another fpga_object object. If a handle object is used when opening the object, then the object is opened with read-write access. Otherwise, the object is read-only.

struct fpga_version
#include <types.h>

Semantic version

Data structure for expressing version identifiers following the semantic versioning scheme. Used in various properties for tracking component versions.

Public Members

uint8_t major

Major version

uint8_t minor

Minor version

uint16_t patch

Revision or patchlevel

struct fpga_error_info
#include <types.h>

Public Members

char name[64]
bool can_clear

name of the error

union metric_value
#include <types.h>

Metric value union

Public Members

uint64_t ivalue
double dvalue
float fvalue
bool bvalue
struct fpga_metric_info
#include <types.h>

Metric info struct

Public Members

uint64_t metric_num
fpga_guid metric_guid
char qualifier_name[256]
char group_name[256]
char metric_name[256]
char metric_units[256]
fpga_metric_datatype metric_datatype
fpga_metric_type metric_type
struct fpga_metric
#include <types.h>

Metric struct

Public Members

uint64_t metric_num
metric_value value

types_enum.h

Definitions of enumerated types for the OPAE C API.

This file defines return and error codes, event and object types, states, and flags as used or reported by OPAE C API functions.

Enums

enum fpga_result

OPAE C API function return codes

Every public API function exported by the OPAE C library will return one of these codes. Usually, FPGA_OK denotes successful completion of the requested operation, while any return code other than FPGA_OK indicates an error or other deviation from the expected behavior. Users of the OPAE C API should always check the return codes of the APIs they call, and not use output parameters of functions that did not execute successfully.

The fpgaErrStr() function converts error codes into printable messages.

OPAE also has a logging mechanism that allows a developer to get more information about why a particular call failed with a specific message. If enabled, any function that returns an error code different from FPGA_OK will also print out a message with further details. This mechanism can be enabled by setting the environment variable LIBOPAE_LOG to 1 before running the respective application.

Values:

FPGA_OK = 0

Operation completed successfully

FPGA_INVALID_PARAM

Invalid parameter supplied

FPGA_BUSY

Resource is busy

FPGA_EXCEPTION

An exception occurred

FPGA_NOT_FOUND

A required resource was not found

FPGA_NO_MEMORY

Not enough memory to complete operation

FPGA_NOT_SUPPORTED

Requested operation is not supported

FPGA_NO_DRIVER

Driver is not loaded

FPGA_NO_DAEMON

FPGA Daemon (fpgad) is not running

FPGA_NO_ACCESS

Insufficient privileges or permissions

FPGA_RECONF_ERROR

Error while reconfiguring FPGA

enum fpga_event_type

FPGA events

OPAE currently defines the following event types that applications can register for. Note that not all FPGA resources and target platforms may support all event types.

Values:

FPGA_EVENT_INTERRUPT = 0

Interrupt generated by an accelerator

FPGA_EVENT_ERROR

Infrastructure error event

FPGA_EVENT_POWER_THERMAL

Infrastructure thermal event

enum fpga_accelerator_state

accelerator state

Values:

FPGA_ACCELERATOR_ASSIGNED = 0

accelerator is opened exclusively by another process

FPGA_ACCELERATOR_UNASSIGNED

accelerator is free to be opened

enum fpga_objtype

OPAE FPGA resources (objects)

These are the FPGA resources currently supported by the OPAE object model.

Values:

FPGA_DEVICE = 0

FPGA_DEVICE objects represent FPGA devices and their management functionality. These objects can be opened (typically requires a certain privilege level or access permissions) and used for management functions like fpgaReconfigreSlot().

FPGA_ACCELERATOR

FPGA_ACCELERATOR objects represent allocatable units for accessing accelerated functions on the FPGA. They are frequently opened for interacting via control registers (MMIO), shared memory, or other, possibly platform-specific functions.

enum fpga_buffer_flags

Buffer flags

These flags can be passed to the fpgaPrepareBuffer() function.

Values:

FPGA_BUF_PREALLOCATED = (1u << 0)

Use existing buffer

FPGA_BUF_QUIET = (1u << 1)

Suppress error messages

enum fpga_open_flags

Open flags

These flags can be passed to the fpgaOpen() function.

Values:

FPGA_OPEN_SHARED = (1u << 0)

Open FPGA resource for shared access

enum fpga_reconf_flags

Reconfiguration flags

These flags can be passed to the fpgaReconfigureSlot() function.

Values:

FPGA_RECONF_FORCE = (1u << 0)

Reconfigure the slot without checking if it is in use

enum fpga_sysobject_flags

Values:

FPGA_OBJECT_SYNC = (1u << 0)

Synchronize data from driver

FPGA_OBJECT_GLOB = (1u << 1)

Treat names as glob expressions

FPGA_OBJECT_RAW = (1u << 2)

Read or write object data as raw bytes

FPGA_OBJECT_RECURSE_ONE = (1u << 3)

Create subobjects one level down from containers

FPGA_OBJECT_RECURSE_ALL = (1u << 4)

Create subobjects all levels from from containers

enum fpga_metric_type

fpga metrics types opae defines power,thermal, performance counter and afu metric types

Values:

FPGA_METRIC_TYPE_POWER
FPGA_METRIC_TYPE_THERMAL
FPGA_METRIC_TYPE_PERFORMANCE_CTR
FPGA_METRIC_TYPE_AFU
FPGA_METRIC_TYPE_UNKNOWN
enum fpga_metric_datatype

Metrics data type

Values:

FPGA_METRIC_DATATYPE_INT
FPGA_METRIC_DATATYPE_FLOAT
FPGA_METRIC_DATATYPE_DOUBLE
FPGA_METRIC_DATATYPE_BOOL
FPGA_METRIC_DATATYPE_UNKNOWN

Enumeration API

The OPAE enumeration API allows selective discovery of FPGA resources. When enumerating resources, a list of filter criteria can be passed to the respective function to select a subset of all resources in the system. The fpgaEnumerate() function itself then returns a list of fpga_tokens denoting resources, which can be used in subsequent API calls.

Filter criteria are specified using one or more fpga_properties object. These objects need to be created using fpgaGetProperties() (defined in <opae/properties/h>) before being passed to fpgaEnumerate(). Individual attributes of an fpga_properties object are set using specific accessors, which are also defined in <opae/properties.h>.

enum.h

APIs for resource enumeration and managing tokens.

These APIs are the first step for any application using OPAE to discover resources that are present on the system. They allow selective enumeration (i.e. getting a list of resources that match a given list of criteria) and methods to manage the lifecycle of tokens generated by fpgaEnumerate().

Functions

fpga_result fpgaEnumerate(const fpga_properties *filters, uint32_t num_filters, fpga_token *tokens, uint32_t max_tokens, uint32_t *num_matches)

Enumerate FPGA resources present in the system

This call allows the user to query the system for FPGA resources that match a certain set of criteria, e.g. all accelerators that are assigned to a host interface and available, all FPGAs of a specific type, etc.

fpgaEnumerate() will create a number of fpga_tokens to represent the matching resources and populate the array tokens with these tokens. The max_tokens argument can be used to limit the number of tokens allocated/returned by fpgaEnumerate(); i.e., the number of tokens in the returned tokens array will be either max_tokens or num_matches (the number of resources matching the filter), whichever is smaller. Use fpgaDestroyToken() to destroy tokens that are no longer needed.

To query the number of matches for a particular set of filters (e.g. to allocate a tokens array of the appropriate size), call fpgaEnumerate() with the parameter tokens set to NULL; this will only return the number of matches in num_matches.

Note
fpgaEnumerate() will allocate memory for the created tokens returned in tokens. It is the responsibility of the using application to free this memory after use by calling fpgaDestroyToken() for each of the returned tokens.
Return
FPGA_OK on success. FPGA_INVALID_PARAM if invalid pointers or objects are passed into the function. FPGA_NO_DRIVER if OPAE can’t find the respective enumeration data structures usually provided by the driver. FPGA_NO_MEMORY if there was not enough memory to create tokens.
Parameters
  • filters: Array of fpga_properties objects describing the properties of the objects that should be returned. A resource is considered matching if its properties match any one of the supplied filters. To match all FPGA resources, pass an empty filters object (one without any filter criteria set) or pass a NULL filters parameter with num_filters set to 0.
  • num_filters: Number of entries in the filters array, or 0 to match all FPGA resources when filters is NULL.
  • tokens: Pointer to an array of fpga_token variables to be populated. If NULL is supplied, fpgaEnumerate() will not create any tokens, but it will return the number of possible matches in num_match.
  • max_tokens: Maximum number of tokens that fpgaEnumerate() shall return (length of tokens array). There may be more or fewer matches than this number; num_matches is set to the number of actual matches.
  • num_matches: Number of resources matching the filter criteria. This number can be higher than the number of tokens returned in the tokens array (depending on the value of max_tokens).

fpga_result fpgaCloneToken(fpga_token src, fpga_token *dst)

Clone a fpga_token object

Creates a copy of an fpga_token object.

Note
This call creates a new token object and allocates memory for it. It is the responsibility of the using application to free this memory after use by calling fpgaDestroyToken() for the cloned token.
Return
FPGA_OK on success
Parameters
  • src: fpga_token object to copy
  • dst: New fpga_token object cloned from ‘src’

fpga_result fpgaDestroyToken(fpga_token *token)

Destroy a Token

This function destroys a token created by fpgaEnumerate() and frees the associated memory.

Note
fpgaDestroyToken() requires the address of an fpga_token as previously created by fpgaEnumerate() or fpgaCloneToken(). Passing any other value results in undefined behavior.
Return
FPGA_OK on success
Parameters
  • token: fpga_token to destroy

properties.h

Functions for examining and manipulating fpga_properties objects.

In OPAE, fpga_properties objects are used both for obtaining information about resources and for selectively enumerating resources based on their properties. This file provides accessor functions (get/set) to allow reading and writing individual items of an fpga_properties object. Generally, not all object types supported by OPAE carry all properties. If you call a property accessor method on a fpga_properties object that does not support this particular property, it will return FPGA_INVALID_PARAM.

Accessor Return Values

In addition to the return values specified in the documentation below, all accessor functions return FPGA_OK on success, FPGA_INVALID_PARAM if you pass NULL or invalid parameters (i.e. non-initialized properties objects), FPGA_EXCEPTION if an internal exception occurred trying to access the properties object, FPGA_NOT_FOUND if the requested property is not part of the supplied properties object.

Functions

fpga_result fpgaGetPropertiesFromHandle(fpga_handle handle, fpga_properties *prop)

Create a fpga_properties object

Initializes the memory pointed at by prop to represent a properties object, and populates it with the properties of the resource referred to by handle. Individual properties can then be queried using fpgaPropertiesGet*() accessor functions.

Note
fpgaGetPropertiesFromHandle() will allocate memory for the created properties object returned in prop. It is the responsibility of the caller to free this memory after use by calling fpgaDestroyProperties().
Return
FPGA_OK on success. FPGA_NO_MEMORY if no memory could be allocated to create the fpga_properties object. FPGA_EXCEPTION if an exception happend while initializing the fpga_properties object.
Parameters
  • handle: Open handle to get properties for.
  • prop: Pointer to a variable of type fpga_properties

fpga_result fpgaGetProperties(fpga_token token, fpga_properties *prop)

Create a fpga_properties object

Initializes the memory pointed at by prop to represent a properties object, and populates it with the properties of the resource referred to by token. Individual properties can then be queried using fpgaPropertiesGet*() accessor functions.

If token is NULL, an “empty” properties object is created to be used as a filter for fpgaEnumerate(). All individual fields are set to dont care`, which implies that the fpga_properties object would match all FPGA resources if used for an fpgaEnumerate() query. The matching criteria can be further refined by using fpgaSet* functions on the properties object, or the object can be populated with the actual properties of a resource by using fpgaUpdateProperties().

Note
fpgaGetProperties() will allocate memory for the created properties object returned in prop. It is the responsibility of the caller to free this memory after use by calling fpgaDestroyProperties().
Return
FPGA_OK on success. FPGA_NO_MEMORY if no memory could be allocated to create the fpga_properties object. FPGA_EXCEPTION if an exception happend while initializing the fpga_properties object.
Parameters
  • token: Token to get properties for. Can be NULL, which will create an empty properties object to be used as a filter for fpgaEnumerate().
  • prop: Pointer to a variable of type fpga_properties

fpga_result fpgaUpdateProperties(fpga_token token, fpga_properties prop)

Update a fpga_properties object

Populates the properties object ‘prop’ with properties of the resource referred to by ‘token’. Unlike fpgaGetProperties(), this call will not create a new properties object or allocate memory for it, but use a previously created properties object.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if token or prop are not valid objects. FPGA_NOT_FOUND if the resource referred to by token was not found. FPGA_NO_DRIVER if not driver is loaded. FPGA_EXCEPTION if an internal exception occured when trying to update prop.
Parameters
  • token: Token to retrieve properties for
  • prop: fpga_properties object to update

fpga_result fpgaClearProperties(fpga_properties prop)

Clear a fpga_properties object

Sets all fields of the properties object pointed at by ‘prop’ to ‘don’t care’, which implies that the fpga_properties object would match all FPGA resources if used for an fpgaEnumerate() query. The matching criteria can be further refined by using fpgaSet* functions on the properties object.

Instead of creating a new fpga_properties object every time, this function can be used to re-use fpga_properties objects from previous queries.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if prop is not a valid object. FPGA_EXCEPTION if an * internal exception occured when trying to access prop.
Parameters
  • prop: fpga_properties object to clear

fpga_result fpgaCloneProperties(fpga_properties src, fpga_properties *dst)

Clone a fpga_properties object

Creates a copy of an fpga_properties object.

Note
This call creates a new properties object and allocates memory for it. Both the ‘src’ and the newly created ‘dst’ objects will eventually need to be destroyed using fpgaDestroyProperties().
Return
FPGA_OK on success. FPGA_INVALID_PARAM if src is not a valid object, or if dst is NULL. FPGA_NO_MEMORY if there was not enough memory to allocate an fpga_properties object for dst. FPGA_EXCEPTION if an internal exception occurred either accessing src or updating dst.
Parameters
  • src: fpga_properties object to copy
  • dst: New fpga_properties object cloned from ‘src’

fpga_result fpgaDestroyProperties(fpga_properties *prop)

Destroy a fpga_properties object

Destroys an existing fpga_properties object that the caller has previously created using fpgaGetProperties() or fpgaCloneProperties().

Note
fpgaDestroyProperties() requires the address of an fpga_properties object, similar to fpgaGetPropertiesFromHandle(), fpgaGetProperties(), and fpgaCloneProperties(). Passing any other value results in undefined behavior.
Return
FPGA_OK on success. FPGA_INVALID_PARAM is prop is not a valid object. FPGA_EXCEPTION if an internal exception occurrred while trying to access prop.
Parameters
  • prop: Pointer to the fpga_properties object to destroy

fpga_result fpgaPropertiesGetParent(const fpga_properties prop, fpga_token *parent)

Get the token of the parent object

Returns the token of the parent of the queried resource in ‘*parent’.

Return
FPGA_NOT_FOUND if resource does not have a parent (e.g. an FPGA_DEVICE resource does not have parents). Also see “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • parent: Pointer to a token variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetParent(fpga_properties prop, fpga_token parent)

Set the token of the parent object

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • parent: Pointer to a token variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetObjectType(const fpga_properties prop, fpga_objtype *objtype)

Get the object type of a resource

Returns the object type of the queried resource.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • objtype: Pointer to an object type variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetObjectType(fpga_properties prop, fpga_objtype objtype)

Set the object type of a resource

Sets the object type of the resource. * Currently supported object types are FPGA_DEVICE and FPGA_ACCELERATOR.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • objtype: Object type of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetSegment(const fpga_properties prop, uint16_t *segment)

Get the PCI segment number of a resource

Returns the segment number of the queried resource.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • segment: Pointer to a PCI segment variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetSegment(fpga_properties prop, uint16_t segment)

Set the PCI segment number of a resource

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • segment: PCI segment number of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetBus(const fpga_properties prop, uint8_t *bus)

Get the PCI bus number of a resource

Returns the bus number the queried resource.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • bus: Pointer to a PCI bus variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetBus(fpga_properties prop, uint8_t bus)

Set the PCI bus number of a resource

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • bus: PCI bus number of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetDevice(const fpga_properties prop, uint8_t *device)

Get the PCI device number of a resource

Returns the device number the queried resource.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • device: Pointer to a PCI device variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetDevice(fpga_properties prop, uint8_t device)

Set the PCI device number of a resource

Enforces the limitation on the number of devices as specified in the PCI spec.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • device: PCI device number of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetFunction(const fpga_properties prop, uint8_t *function)

Get the PCI function number of a resource

Returns the function number the queried resource.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • function: Pointer to PCI function variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetFunction(fpga_properties prop, uint8_t function)

Set the PCI function number of a resource

Enforces the limitation on the number of functions as specified in the PCI spec.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • function: PCI function number of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetSocketID(const fpga_properties prop, uint8_t *socket_id)

Get the socket id of a resource

Returns the socket id of the queried resource.

Return
See “Accessor Return Values” in properties.h. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • socket_id: Pointer to a socket id variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetSocketID(fpga_properties prop, uint8_t socket_id)

Set the socket id of the resource

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • socket_id: Socket id of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetDeviceID(const fpga_properties prop, uint16_t *device_id)

Get the device id of the resource

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • device_id: Pointer to a device id variable of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesSetDeviceID(fpga_properties prop, uint16_t device_id)

Set the device id of the resource

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • device_id: Device id of the resource ‘prop’ is associated with

fpga_result fpgaPropertiesGetNumSlots(const fpga_properties prop, uint32_t *num_slots)

Get the number of slots of an FPGA resource property

Returns the number of slots present in an FPGA.

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • num_slots: Pointer to number of slots variable of the FPGA

fpga_result fpgaPropertiesSetNumSlots(fpga_properties prop, uint32_t num_slots)

Set the number of slots of an FPGA resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • num_slots: Number of slots of the FPGA

fpga_result fpgaPropertiesGetBBSID(const fpga_properties prop, uint64_t *bbs_id)

Get the BBS ID of an FPGA resource property

Returns the blue bitstream id of an FPGA.

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • bbs_id: Pointer to a bbs id variable of the FPGA

fpga_result fpgaPropertiesSetBBSID(fpga_properties prop, uint64_t bbs_id)

Set the BBS ID of an FPGA resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • bbs_id: Blue bitstream id of the FPGA resource

fpga_result fpgaPropertiesGetBBSVersion(const fpga_properties prop, fpga_version *bbs_version)

Get the BBS Version of an FPGA resource property

Returns the blue bitstream version of an FPGA.

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • bbs_version: Pointer to a bbs version variable of the FPGA

fpga_result fpgaPropertiesSetBBSVersion(fpga_properties prop, fpga_version version)

Set the BBS Version of an FPGA resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • version: Blue bitstream version of the FPGA resource

fpga_result fpgaPropertiesGetVendorID(const fpga_properties prop, uint16_t *vendor_id)

Get the vendor id of an FPGA resource property

Returns the vendor id of an FPGA.

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • vendor_id: Pointer to a vendor id variable of the FPGA

fpga_result fpgaPropertiesSetVendorID(fpga_properties prop, uint16_t vendor_id)

Set the vendor id of an FPGA resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • vendor_id: Vendor id of the FPGA resource

fpga_result fpgaPropertiesGetModel(const fpga_properties prop, char *model)

Get the model of an FPGA resource property

Returns the model of an FPGA.

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • model: Model of the FPGA resource (string of minimum FPGA_MODEL_LENGTH length

fpga_result fpgaPropertiesSetModel(fpga_properties prop, char *model)

Set the model of an FPGA resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • model: Model of the FPGA resource (string of maximum FPGA_MODEL_LENGTH length

fpga_result fpgaPropertiesGetLocalMemorySize(const fpga_properties prop, uint64_t *lms)

Get the local memory size of an FPGA resource property

Returns the local memory size of an FPGA.

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • lms: Pointer to a memory size variable of the FPGA

fpga_result fpgaPropertiesSetLocalMemorySize(fpga_properties prop, uint64_t lms)

Set the local memory size of an FPGA resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • lms: Local memory size of the FPGA resource

fpga_result fpgaPropertiesGetCapabilities(const fpga_properties prop, uint64_t *capabilities)

Get the capabilities FPGA resource property

Returns the capabilities of an FPGA. Capabilities is a bitfield value

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to query - must be of type FPGA_DEVICE
  • capabilities: Pointer to a capabilities variable of the FPGA

fpga_result fpgaPropertiesSetCapabilities(fpga_properties prop, uint64_t capabilities)

Set the capabilities of an FPGA resource property

Capabilities is a bitfield value

Return
FPGA_INVALID_PARAM if object type is not FPGA_DEVICE. See also “Accessor Return Values” in properties.h.
Note
This API is not currently supported.
Parameters
  • prop: Properties object to modify - must be of type FPGA_DEVICE
  • capabilities: Capabilities of the FPGA resource

fpga_result fpgaPropertiesGetGUID(const fpga_properties prop, fpga_guid *guid)

Get the GUID of a resource

Returns the GUID of an FPGA or accelerator object.

For an accelerator, the GUID uniquely identifies a specific accelerator context type, i.e. different accelerators will have different GUIDs. For an FPGA, the GUID is used to identify a certain instance of an FPGA, e.g. to determine whether a given bitstream would be compatible.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • guid: Pointer to a GUID of the slot variable

fpga_result fpgaPropertiesSetGUID(fpga_properties prop, fpga_guid guid)

Set the GUID of a resource

Sets the GUID of an FPGA or accelerator object.

For an accelerator, the GUID uniquely identifies a specific accelerator context type, i.e. different accelerators will have different GUIDs. For an FPGA, the GUID is used to identify a certain instance of an FPGA, e.g. to determine whether a given bitstream would be compatible.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify
  • guid: Pointer to a GUID of the slot variable

fpga_result fpgaPropertiesGetNumMMIO(const fpga_properties prop, uint32_t *mmio_spaces)

Get the number of mmio spaces

Returns the number of mmio spaces of an AFU properties structure.

Return
FPGA_INVALID_PARAM if object type is not FPGA_ACCELERATOR. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query - must be of type FPGA_ACCELERATOR
  • mmio_spaces: Pointer to a variable for number of mmio spaces

fpga_result fpgaPropertiesSetNumMMIO(fpga_properties prop, uint32_t mmio_spaces)

Set the number of mmio spaces

Sets the number of mmio spaces of an AFU properties structure.

Return
FPGA_INVALID_PARAM if object type is not FPGA_ACCELERATOR. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify - must be of type FPGA_ACCELERATOR
  • mmio_spaces: Number of MMIO spaces of the accelerator

fpga_result fpgaPropertiesGetNumInterrupts(const fpga_properties prop, uint32_t *num_interrupts)

Get the number of interrupts

Returns the number of interrupts of an accelerator properties structure.

Return
FPGA_INVALID_PARAM if object type is not FPGA_ACCELERATOR. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query - must be of type FPGA_ACCELERATOR
  • num_interrupts: Pointer to a variable for number of interrupts

fpga_result fpgaPropertiesSetNumInterrupts(fpga_properties prop, uint32_t num_interrupts)

Set the number of interrupts

Sets the number of interrupts of an accelerator properties structure.

Return
FPGA_INVALID_PARAM if object type is not FPGA_ACCELERATOR. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify - must be of type FPGA_ACCELERATOR
  • num_interrupts: Number of interrupts of the accelerator

fpga_result fpgaPropertiesGetAcceleratorState(const fpga_properties prop, fpga_accelerator_state *state)

Get the state of a accelerator resource property

Returns the accelerator state of a accelerator.

Return
FPGA_INVALID_PARAM if object type is not FPGA_ACCELERATOR. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query - must be of type FPGA_ACCELERATOR
  • state: Pointer to a accelerator state variable of the accelerator

fpga_result fpgaPropertiesSetAcceleratorState(fpga_properties prop, fpga_accelerator_state state)

Set the state of an accelerator resource property

Return
FPGA_INVALID_PARAM if object type is not FPGA_ACCELERATOR. See also “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to modify - must be of type FPGA_ACCELERATOR
  • state: accelerator state of the accelerator resource

fpga_result fpgaPropertiesGetObjectID(const fpga_properties prop, uint64_t *object_id)

Get the object ID of a resource

Returns the object ID of a resource. The object ID is a 64 bit identifier that is unique within a single node or system. It represents a similar concept as the token, but can be used across processes (e.g. passed on the command line).

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • object_id: Pointer to a 64bit memory location to store the object ID in

fpga_result fpgaPropertiesSetObjectID(const fpga_properties prop, uint64_t object_id)

Set the object ID of a resource

Sets the object ID of a resource. The object ID is a 64 bit identifier that is unique within a single node or system. It represents a similar concept as the token, but can be used across processes (e.g. passed on the command line).

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • object_id: A 64bit value to use as the object ID

fpga_result fpgaPropertiesGetNumErrors(const fpga_properties prop, uint32_t *num_errors)

Get the number of errors that can be reported by a resource

Returns the number of error registers understood by a resource.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • num_errors: Pointer to a 32 bit memory location to store the number of supported errors in

fpga_result fpgaPropertiesSetNumErrors(const fpga_properties prop, uint32_t num_errors)

Set the number of error registers

Set the number of error registers understood by a resource to enumerate.

Return
See “Accessor Return Values” in properties.h.
Parameters
  • prop: Properties object to query
  • num_errors: Number of errors

Access API

The access API provides functions for opening and closing FPGA resources. Opening a resource yields an fpga_handle, which denotes ownership and can be used in subsequent API calls to interact with a specific resource. Ownership can be exclusive or shared.

access.h

Functions to acquire, release, and reset OPAE FPGA resources.

Functions

fpga_result fpgaOpen(fpga_token token, fpga_handle *handle, int flags)

Open an FPGA object

Acquires ownership of the FPGA resource referred to by ‘token’.

Most often this will be used to open an accelerator object to directly interact with an accelerator function, or to open an FPGA object to perform management functions.

Return
FPGA_OK on success. FPGA_NOT_FOUND if the resource for ‘token’ could not be found. FPGA_INVALID_PARAM if ‘token’ does not refer to a resource that can be opened, or if either argument is NULL or invalid. FPGA_EXCEPTION if an internal exception occurred while creating the handle. FPGA_NO_DRIVER if the driver is not loaded. FPGA_BUSY if trying to open a resource that has already been opened in exclusive mode. FPGA_NO_ACCESS if the current process’ privileges are not sufficient to open the resource.
Parameters
  • token: Pointer to token identifying resource to acquire ownership of
  • handle: Pointer to preallocated memory to place a handle in. This handle will be used in subsequent API calls.
  • flags: One of the following flags:
    • FPGA_OPEN_SHARED allows the resource to be opened multiple times (not supported in ASE) Shared resources (including buffers) are released when all associated handles have been closed (either explicitly with fpgaClose() or by process termination).

fpga_result fpgaClose(fpga_handle handle)

Close a previously opened FPGA object

Relinquishes ownership of a previously fpgaOpen()ed resource. This enables others to acquire ownership if the resource was opened exclusively. Also deallocates / unmaps MMIO and UMsg memory areas.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if handle does not refer to an acquired resource, or if handle is NULL. FPGA_EXCEPTION if an internal error occurred while accessing the handle.
Parameters
  • handle: Handle to previously opened FPGA object

fpga_result fpgaReset(fpga_handle handle)

Reset an FPGA object

Performs an accelerator reset.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if handle does not refer to an acquired resource or to a resoure that cannot be reset. FPGA_EXCEPTION if an internal error occurred while trying to access the handle or resetting the resource.
Parameters
  • handle: Handle to previously opened FPGA object

Event API

The event API provides functions and types for handling asynchronous events such as errors or AFC interrupts.

To natively support asynchronous event, the driver for the FPGA platform needs to support events natively (in which case the OPAE C library will register the event directly with the driver). For some platforms that do not support interrupt-driven event delivery, you need to run the FPGA Daemon (fpgad) to enable asynchronous OPAE events. fpgad will act as a proxy for the application and deliver asynchronous notifications for registered events.

event.h

Functions for registering events and managing the lifecycle for fpga_event_handles.

OPAE provides an interface to asynchronous events that can be generated by different FPGA resources. The event API provides functions to register for these events; associated with every event a process has registered for is an fpga_event_handle, which encapsulates the OS-specific data structure for event objects. On Linux, an fpga_event_handle can be used as a file descriptor and passed to select(), poll(), epoll() and similar functions to wait for asynchronous events.

Functions

fpga_result fpgaCreateEventHandle(fpga_event_handle *event_handle)

Initialize an event_handle

Platform independent way to initialize an event_handle used for notifications from the driver to application. For Linux, this function creates an eventfd and returns the eventfd file descriptor in *event_handle.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if event_handle is NULL. FPGA_NOT_SUPPORTED if platform does not support events.
Parameters
  • event_handle: Pointer to event handle variable.

fpga_result fpgaDestroyEventHandle(fpga_event_handle *event_handle)

Destroy an event_handle

Destroy handle and free resources. On Linux this corresponds to closing the file descriptor pointed to by handle

Note
fpgaDestroyEventHandle() requires the address of an event_handle as created by fpgaCreateEventHandle(). Passing any other value results in undefined behavior.
Return
FPGA_OK on success. FPGA_INVALID_PARAM if event_handle is NULL.
Parameters
  • event_handle: Pointer to handle to be destroyed

fpga_result fpgaGetOSObjectFromEventHandle(const fpga_event_handle eh, int *fd)

Get OS object from event handle

Check validity of event handle, and get the OS object used to subscribe and unsubscribe to events. On Linux, the object corresponds to a file descriptor.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if event_handle is invalid.
Parameters
  • eh: Event handle to get the descriptor value from
  • fd: integer to store the descriptor value

fpga_result fpgaRegisterEvent(fpga_handle handle, fpga_event_type event_type, fpga_event_handle event_handle, uint32_t flags)

Register an FPGA event

This function tells the driver that the caller is interested in notification for the event specified by the type and flags pair.

The event_handle points to an OS specific mechanism for event notification. An event_handle is associated with only a single event.

In case of user interrupts, the flags parameter will be used to specify the vector ID. The value of the flags parameter indicates the vector ID, no bit encoding is used.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if handle does not refer to a resource supporting the requested event, or if event_handle is not valid. FPGA_EXCEPTION if an internal exception occurred while accessing the handle or the event_handle. On Linux: FPGA_NO_DAEMON if the driver does not support the requested event and there is no FPGA Daemon (fpgad) running to proxy it.
Parameters
  • handle: Handle to previously opened FPGA resource.
  • event_type: Type of event
  • event_handle: Handle to previously opened resource for event notification.
  • flags: Optional argument for specifying additional information about event. For example irq number for interrupt events.

fpga_result fpgaUnregisterEvent(fpga_handle handle, fpga_event_type event_type, fpga_event_handle event_handle)

Unregister an FPGA event

This function tells the driver that the caller is no longer interested in notification for the event associated with the event_handle

The event_handle points to an OS specific mechanism for event notification. An event_handle is associated with only a single event.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if handle does not refer to a resource supporting the requested event, or if event_handle is not valid. FPGA_EXCEPTION if an internal error occurred accessing the handle or the event_handle.
Parameters
  • handle: Handle to previously opened FPGA resource.
  • event_type: Type of event to unregister.
  • event_handle: Handle to previously registered resource for event notification.

MMIO and Shared Memory APIs

These APIs feature functions for mapping and accessing control registers through memory-mapped IO (mmio.h), allocating and sharing system memory buffers with an accelerator (buffer.h), and using low-latency notifications (umsg.h).

mmio.h

Functions for mapping and accessing MMIO space.

Most FPGA accelerators provide access to control registers through memory-mappable address spaces, commonly referred to as “MMIO spaces”. This file provides functions to map, unmap, read, and write MMIO spaces.

Note that an accelerator may have multiple MMIO spaces, denoted by the mmio_num argument of the APIs below. The meaning and properties of each MMIO space are up to the accelerator designer.

Functions

fpga_result fpgaWriteMMIO64(fpga_handle handle, uint32_t mmio_num, uint64_t offset, uint64_t value)

Write 64 bit value to MMIO space

This function will write to MMIO space of the target object at a specified offset.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Parameters
  • handle: Handle to previously opened accelerator resource
  • mmio_num: Number of MMIO space to access
  • offset: Byte offset into MMIO space
  • value: Value to write (64 bit)

fpga_result fpgaReadMMIO64(fpga_handle handle, uint32_t mmio_num, uint64_t offset, uint64_t *value)

Read 64 bit value from MMIO space

This function will read from MMIO space of the target object at a specified offset.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Parameters
  • handle: Handle to previously opened accelerator resource
  • mmio_num: Number of MMIO space to access
  • offset: Byte offset into MMIO space
  • value: Pointer to memory where read value is returned (64 bit)

fpga_result fpgaWriteMMIO32(fpga_handle handle, uint32_t mmio_num, uint64_t offset, uint32_t value)

Write 32 bit value to MMIO space

This function will write to MMIO space of the target object at a specified offset.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Parameters
  • handle: Handle to previously opened accelerator resource
  • mmio_num: Number of MMIO space to access
  • offset: Byte offset into MMIO space
  • value: Value to write (32 bit)

fpga_result fpgaReadMMIO32(fpga_handle handle, uint32_t mmio_num, uint64_t offset, uint32_t *value)

Read 32 bit value from MMIO space

This function will read from MMIO space of the target object at a specified offset.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Parameters
  • handle: Handle to previously opened accelerator resource
  • mmio_num: Number of MMIO space to access
  • offset: Byte offset into MMIO space
  • value: Pointer to memory where read value is returned (32 bit)

fpga_result fpgaMapMMIO(fpga_handle handle, uint32_t mmio_num, uint64_t **mmio_ptr)

Map MMIO space

This function will return a pointer to the specified MMIO space of the target object in process virtual memory, if supported by the target. Some MMIO spaces may be restricted to privileged processes, depending on the used handle and type.

After mapping the respective MMIO space, you can access it through direct pointer operations (observing supported access sizes and alignments of the target platform and accelerator).

If the caller passes in NULL for mmio_ptr, no mapping will be performed, and no virtual address will be returned, though the call will return FPGA_OK. This implies that all accesses will be performed through fpgaReadMMIO32(), fpgaWriteMMIO32(), fpgeReadMMIO64(), and fpgaWriteMMIO64(). This is the only supported case for ASE.

Note
Some targets (such as the ASE simulator) do not support memory-mapping of IO register spaces and will not return a pointer to an actually mapped space. Instead, they will return FPGA_NOT_SUPPORTED. Usually, these platforms still allow the application to issue MMIO operations using fpgaReadMMIO32(), fpgaWriteMMIO32(), fpgeReadMMIO64(), and fpgaWriteMMIO64().

The number of available MMIO spaces can be retrieved through the num_mmio property (fpgaPropertyGetNumMMIO()).

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle. FPGA_NO_ACCESS if the process’ permissions are not sufficient to map the requested MMIO space. FPGA_NOT_SUPPORTED if platform does not support memory mapped IO.
Parameters
  • handle: Handle to previously opened resource
  • mmio_num: Number of MMIO space to access
  • mmio_ptr: Pointer to memory where a pointer to the MMIO space will be returned. May be NULL, in which case no pointer is returned. Returned address may be NULL if underlying platform does not support memory mapping for register access.

fpga_result fpgaUnmapMMIO(fpga_handle handle, uint32_t mmio_num)

Unmap MMIO space

This function will unmap a previously mapped MMIO space of the target opject, rendering any pointers to it invalid.

Note
This call is only supported by hardware targets, not by ASE simulation.
Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Parameters
  • handle: Handle to previously opened resource
  • mmio_num: Number of MMIO space to access

buffer.h

Functions for allocating and sharing system memory with an FPGA accelerator.

To share memory between a software application and an FPGA accelerator, these functions set up system components (e.g. an IOMMU) to allow accelerator access to a provided memory region.

There are a number of restrictions on what memory can be shared, depending on platform capabilities. Usually, FPGA accelerators to not have access to virtual address mappings of the CPU, so they can only access physical addresses. To support this, the OPAE C library on Linux uses hugepages to allocate large, contiguous pages of physical memory that can be shared with an accalerator. It also supports sharing memory that has already been allocated by an application, as long as that memory satisfies the requirements of being physically contigous and page-aligned.

Functions

fpga_result fpgaPrepareBuffer(fpga_handle handle, uint64_t len, void **buf_addr, uint64_t *wsid, int flags)

Prepare a shared memory buffer

Prepares a memory buffer for shared access between an accelerator and the calling process. This may either include allocation of physical memory, or preparation of already allocated memory for sharing. The latter case is indicated by supplying the FPGA_BUF_PREALLOCATED flag.

This function will ask the driver to pin the indicated memory (make it non-swappable), and program the IOMMU to allow access from the accelerator. If the buffer was not pre-allocated (flag FPGA_BUF_PREALLOCATED), the function will also allocate physical memory of the requested size and map the memory into the caller’s process’ virtual address space. It returns in ‘wsid’ an fpga_buffer object that can be used to program address registers in the accelerator for shared access to the memory.

When using FPGA_BUF_PREALLOCATED, the input len must be a non-zero multiple of the page size, else the function returns FPGA_INVALID_PARAM. When not using FPGA_BUF_PREALLOCATED, the input len is rounded up to the nearest multiple of page size.

Return
FPGA_OK on success. FPGA_NO_MEMORY if the requested memory could not be allocated. FPGA_INVALID_PARAM if invalid parameters were provided, or if the parameter combination is not valid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Note
As a special case, when FPGA_BUF_PREALLOCATED is present in flags, if len == 0 and buf_addr == NULL, then the function returns FPGA_OK if pre-allocated buffers are supported. In this case, a return value other than FPGA_OK indicates that pre-allocated buffers are not supported.
Parameters
  • handle: Handle to previously opened accelerator resource
  • len: Length of the buffer to allocate/prepare in bytes
  • buf_addr: Virtual address of buffer. Contents may be NULL (OS will choose mapping) or non-NULL (OS will take contents as a hint for the virtual address).
  • wsid: Handle to the allocated/prepared buffer to be used with other functions
  • flags: Flags. FPGA_BUF_PREALLOCATED indicates that memory pointed at in ‘*buf_addr’ is already allocated an mapped into virtual memory.

fpga_result fpgaReleaseBuffer(fpga_handle handle, uint64_t wsid)

Release a shared memory buffer

Releases a previously prepared shared buffer. If the buffer was allocated using fpgaPrepareBuffer (FPGA_BUF_PREALLOCATED was not specified), this call will deallocate/free that memory. Otherwise, it will only be returned to it’s previous state (pinned/unpinned, cached/non-cached).

Return
FPGA_OK on success. FPGA_INVALID_PARAM if invalid parameters were provided, or if the parameter combination is not valid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle.
Parameters
  • handle: Handle to previously opened accelerator resource
  • wsid: Handle to the allocated/prepared buffer

fpga_result fpgaGetIOAddress(fpga_handle handle, uint64_t wsid, uint64_t *ioaddr)

Retrieve base IO address for buffer

This function is used to acquire the physical base address (on some platforms called IO Virtual Address or IOVA) for a shared buffer identified by wsid.

Note
This function will disappear once the APIs for secure sharing of buffer addresses is implemented.
Return
FPGA_OK on success. FPGA_INVALID_PARAM if invalid parameters were provided, or if the parameter combination is not valid. FPGA_EXCEPTION if an internal exception occurred while trying to access the handle. FPGA_NOT_FOUND if wsid does not refer to a previously shared buffer.
Parameters
  • handle: Handle to previously opened accelerator resource
  • wsid: Buffer handle / workspace ID referring to the buffer for which the IO address is requested
  • ioaddr: Pointer to memory where the IO address will be returned

umsg.h

FPGA UMsg API.

Functions

fpga_result fpgaGetNumUmsg(fpga_handle handle, uint64_t *value)

Get number of Umsgs

Retuns number of umsg supported by AFU.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if input parameter combination is not valid. FPGA_EXCEPTION if input parameter fpga handle is not valid.
Parameters
  • handle: Handle to previously opened accelerator resource
  • value: Returns number of UMsgs

fpga_result fpgaSetUmsgAttributes(fpga_handle handle, uint64_t value)

Sets Umsg hint

Writes usmg hint bit.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if input parameter combination is not valid. FPGA_EXCEPTION if input parameter fpga handle is not valid.
Parameters
  • handle: Handle to previously opened accelerator resource
  • value: Value to use for UMsg hint, Umsg hit is N wide bitvector where N = number of Umsgs.

fpga_result fpgaTriggerUmsg(fpga_handle handle, uint64_t value)

Trigger Umsg

Writes a 64-bit value to trigger low-latency accelerator notification mechanism (UMsgs).

Return
FPGA_OK on success. FPGA_INVALID_PARAM if input parameter combination is not valid. FPGA_EXCEPTION if input parameter fpga handle is not valid.
Parameters
  • handle: Handle to previously opened accelerator resource
  • value: Value to use for UMsg

fpga_result fpgaGetUmsgPtr(fpga_handle handle, uint64_t **umsg_ptr)

Access UMsg memory directly

This function will return a pointer to the memory allocated for low latency accelerator notifications (UMsgs).

Return
FPGA_OK on success. FPGA_INVALID_PARAM if input parameter combination is not valid. FPGA_EXCEPTION if input parameter fpga handle is not valid. FPGA_NO_MEMORY if memory allocation fails or system doesn’t configure huge pages.
Parameters
  • handle: Handle to previously opened accelerator resource
  • umsg_ptr: Pointer to memory where a pointer to the virtual address space will be returned

Management API

The management APIs define functions for reconfiguring an FPGA (writing new partial bitstreams) as well as assigning accelerators to host interfaces.

manage.h

Functions for managing FPGA configurations.

FPGA accelerators can be reprogrammed at run time by providing new partial bitstreams (“green bitstreams”). This file defines API functions for programming green bitstreams as well as for assigning accelerators to host interfaces for more complex deployment setups, such as virtualized systems.

Functions

fpga_result fpgaAssignPortToInterface(fpga_handle fpga, uint32_t interface_num, uint32_t slot_num, int flags)

Assign Port to a host interface.

This function assign Port to a host interface for subsequent use. Only Port that have been assigned to a host interface can be opened by fpgaOpen().

Return
FPGA_OK on success FPGA_INVALID_PARAM if input parameter combination is not valid. FPGA_EXCEPTION if an exception occcurred accessing the fpga handle. FPGA_NOT_SUPPORTED if driver does not support assignment.
Parameters
  • fpga: Handle to an FPGA object previously opened that both the host interface and the slot belong to
  • interface_num: Host interface number
  • slot_num: Slot number
  • flags: Flags (to be defined)

fpga_result fpgaAssignToInterface(fpga_handle fpga, fpga_token accelerator, uint32_t host_interface, int flags)

Assign an accelerator to a host interface

This function assigns an accelerator to a host interface for subsequent use. Only accelerators that have been assigned to a host interface can be opened by fpgaOpen().

Note
This function is currently not supported.
Return
FPGA_OK on success
Parameters
  • fpga: Handle to an FPGA object previously opened that both the host interface and the accelerator belong to
  • accelerator: accelerator to assign
  • host_interface: Host interface to assign accelerator to
  • flags: Flags (to be defined)

fpga_result fpgaReleaseFromInterface(fpga_handle fpga, fpga_token accelerator)

Unassign a previously assigned accelerator

This function removes the assignment of an accelerator to an host interface (e.g. to be later assigned to a different host interface). As a consequence, the accelerator referred to by token ‘accelerator’ will be reset during the course of this function.

Note
This function is currently not supported.
Return
FPGA_OK on success
Parameters
  • fpga: Handle to an FPGA object previously opened that both the host interface and the accelerator belong to
  • accelerator: accelerator to unassign/release

fpga_result fpgaReconfigureSlot(fpga_handle fpga, uint32_t slot, const uint8_t *bitstream, size_t bitstream_len, int flags)

Reconfigure a slot

Sends a green bitstream file to an FPGA to reconfigure a specific slot. This call, if successful, will overwrite the currently programmed AFU in that slot with the AFU in the provided bitstream.

As part of the reconfiguration flow, all accelerators associated with this slot will be unassigned and reset.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if the provided parameters are not valid. FPGA_EXCEPTION if an internal error occurred accessing the handle or while sending the bitstream data to the driver. FPGA_BUSY if the accelerator for the given slot is in use. FPGA_RECONF_ERROR on errors reported by the driver (such as CRC or protocol errors).
Note
By default, fpgaReconfigureSlot will not allow reconfiguring a slot with an accelerator in use. Add the flag FPGA_RECONF_FORCE to force reconfiguration without checking for accelerators in use.
Parameters
  • fpga: Handle to an FPGA object previously opened
  • slot: Token identifying the slot to reconfigure
  • bitstream: Pointer to memory holding the bitstream
  • bitstream_len: Length of the bitstream in bytes
  • flags: Flags that control behavior of reconfiguration. Value of 0 indicates no flags. FPGA_RECONF_FORCE indicates that the bitstream is programmed into the slot without checking if the resource is currently in use.

Metrics API

The metrics APIs define functions for discovery/enumeration of metrics information and reading metrics values.

metrics.h

Functions for Discover/ Enumerates metrics and retrieves values.

Functions

fpga_result fpgaGetNumMetrics(fpga_handle handle, uint64_t *num_metrics)

Enumerates number of metrics

Return
FPGA_OK on success. FPGA_NOT_FOUND if the Metrics are not discovered
Parameters
  • handle: Handle to previously opened fpga resource
  • num_metrics: Number of metrics are discovered in fpga resource

fpga_result fpgaGetMetricsInfo(fpga_handle handle, fpga_metric_info *metric_info, uint64_t *num_metrics)

Retrieve metrics information

Return
FPGA_OK on success. FPGA_NOT_FOUND if the Metrics are not found. FPGA_NO_MEMORY if there was not enough memory to enumerates metrics.
Parameters
  • handle: Handle to previously opened fpga resource
  • metric_info: Pointer to array of metric info struct user allocates metrics info array
  • num_metrics: Size of metric info array

fpga_result fpgaGetMetricsByIndex(fpga_handle handle, uint64_t *metric_num, uint64_t num_metric_indexes, fpga_metric *metrics)

Retrieve metrics values by index

Return
FPGA_OK on success. FPGA_NOT_FOUND if the Metrics are not found. FPGA_NO_MEMORY if there was not enough memory to enumerates metrics.
Parameters
  • handle: Handle to previously opened fpga resource
  • metric_num: Pointer to array of metric index user allocates metric array
  • num_metric_indexes: Size of metric array
  • metrics: pointer to array of metric struct

fpga_result fpgaGetMetricsByName(fpga_handle handle, char **metrics_names, uint64_t num_metric_names, fpga_metric *metrics)

Retrieve metric values by names

Return
FPGA_OK on success. FPGA_NOT_FOUND if the Metrics are not found
Parameters
  • handle: Handle to previously opened fpga resource
  • metrics_names: Pointer to array of metrics name user allocates metrics name array
  • num_metric_names: Size of metric name array
  • metrics: Pointer to array of metric struct

SysObject

The SysObject API can be used to get system objects by name. Names used with the SysObject API are driver-specific and may not be compatible across plugins and/or drivers. For example, SysObject names used with the xfpga plugin will apply to the OPAE Linux Kernel driver and refer to sysfs nodes under the sysfs tree for the resource used with the SysObject API.

sysobject.h

Functions to read/write from system objects. On Linux systems with the OPAE kernel driver, this is used to access sysfs nodes created by the driver.

Functions

fpga_result fpgaTokenGetObject(fpga_token token, const char *name, fpga_object *object, int flags)

Create an fpga_object data structures. An fpga_object is a handle to an FPGA resource which can be an attribute or register or driver attribute. This object is read-only.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_NOT_FOUND if an object cannot be found with the given key. FPGA_NOT_SUPPORTED if this function is not supported by the current implementation of this API.
Note
Names that begin with ‘.’ or ‘/’ or contain ‘..’ are not allowed and result in FPGA_INVALID_PARAM being returned
Parameters
  • token: Token identifying a resource (accelerator or device)
  • name: A key identifying an object belonging to a resource.
  • object: Pointer to memory to store the object in
  • flags: Control behavior of object identification and creation. FPGA_OBJECT_GLOB is used to indicate that the name should be treated as a globbing expression. FPGA_OBJECT_RECURSE_ONE indicates that subobjects be created for objects one level down from the object identified by name. FPGA_OBJECT_RECURSE_ALL indicates that subobjects be created for all objects below the current object identified by name.

fpga_result fpgaHandleGetObject(fpga_handle handle, const char *name, fpga_object *object, int flags)

Create an fpga_object data structure from a handle. An fpga_object is a handle to an FPGA resource which can be an attribute or register. This object has read/write access..

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid. FPGA_NOT_FOUND if an object cannot be found with the given key. FPGA_NOT_SUPPORTED if this function is not supported by the current implementation of this API.
Note
Names that begin with ‘.’ or ‘/’ or contain ‘..’ are not allowed and result in FPGA_INVALID_PARAM being returned
Parameters
  • handle: Handle identifying a resource (accelerator or device)
  • name: A key identifying an object belonging to a resource.
  • object: Pointer to memory to store the object in
  • flags: Control behavior of object identification and creation FPGA_OBJECT_GLOB is used to indicate that the name should be treated as a globbing expression. FPGA_OBJECT_RECURSE_ONE indicates that subobjects be created for objects one level down from the object identified by name. FPGA_OBJECT_RECURSE_ALL indicates that subobjects be created for all objects below the current object identified by name.

fpga_result fpgaObjectGetObject(fpga_object parent, const char *name, fpga_object *object, int flags)

Create an fpga_object data structure from a parent object. An fpga_object is a handle to an FPGA resource which can be an attribute or a register. If the parent object was created with a handle, then the new object will inherit the handle allowing it to have read-write access to the object data.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of the supplied parameters is invalid - this includes a parent object that is not a container object. FPGA_NOT_FOUND if an object cannot be found with the given key. FPGA_NOT_SUPPORTED if this function is not supported by the current implementation of this API.
Note
Names that begin with ‘.’ or ‘/’ or contain ‘..’ are not allowed and result in FPGA_INVALID_PARAM being returned
Parameters
  • parent: A parent container fpga_object.
  • name: A key identifying a sub-object of the parent container.
  • object: Pointer to memory to store the object in.
  • flags: Control behavior of object identification and creation. FPGA_OBJECT_GLOB is used to indicate that the name should be treated as a globbing expression. FPGA_OBJECT_RECURSE_ONE indicates that subobjects be created for objects one level down from the object identified by name. FPGA_OBJECT_RECURSE_ALL indicates that subobjects be created for all objects below the current object identified by name.

fpga_result fpgaDestroyObject(fpga_object *obj)

Free memory used for the fpga_object data structure.

Note
fpgaDestroyObject() requires the address of an fpga_object as created by fpgaTokenGetObject(), fpgaHandleGetObject(), or fpgaObjectGetObject(). Passing any other value results in undefind behavior.
Return
FPGA_OK on success, FPGA_INVALID_PARAM if the object is NULL, FPGA_EXCEPTION if an internal error is encountered.
Parameters
  • obj: Pointer to the fpga_object instance to destroy

fpga_result fpgaObjectGetSize(fpga_object obj, uint32_t *value, int flags)

Retrieve the size of the object.

Return
FPGA_OK on success. FPGA_INVALID_PARAM if any of supplied paramters is invalid. FPGA_EXCEPTION if error occurred.
Parameters
  • obj: An fpga_object instance.
  • value: Pointer to variable to store size in.
  • flags: Flags that control how the object is read If FPGA_OBJECT_SYNC is used then object will update its buffered copy before retrieving the size.

fpga_result fpgaObjectRead(fpga_object obj, uint8_t *buffer, size_t offset, size_t len, int flags)

Read bytes from an FPGA object.

Return
FPGA_OK on success, FPGA_INVALID_PARAM if any of the supplied parameters is invalid
Parameters
  • obj: An fpga_object instance.
  • buffer: Pointer to a buffer to read bytes into.
  • offset: Byte offset relative to objects internal buffer where to begin reading bytes from.
  • len: The length, in bytes, to read from the object.
  • flags: Flags that control how object is read If FPGA_OBJECT_SYNC is used then object will update its buffered copy before retrieving the data.

fpga_result fpgaObjectRead64(fpga_object obj, uint64_t *value, int flags)

Read a 64-bit value from an FPGA object. The value is assumed to be in string format and will be parsed. See flags below for changing that behavior.

Return
FPGA_OK on success, FPGA_INVALID_PARAM if any of the supplied parameters is invalid
Parameters
  • obj: An fpga_object instance
  • value: Pointer to a 64-bit variable to store the value in
  • flags: Flags that control how the object is read If FPGA_OBJECT_SYNC is used then object will update its buffered copy before retrieving the data. If FPGA_OBJECT_RAW is used, then the data will be read as raw bytes into the uint64_t pointer variable.

fpga_result fpgaObjectWrite64(fpga_object obj, uint64_t value, int flags)

Write 64-bit value to an FPGA object. The value will be converted to string before writing. See flags below for changing that behavior.

Return
FPGA_OK on success, FPGA_INVALID_PARAM if any of the supplied parameters is invalid
Note
The object must have been created using a handle to a resource.
Parameters
  • obj: An fpga_object instance.
  • value: The value to write to the object
  • flags: Flags that control how the object is written If FPGA_OBJECT_RAW is used, then the value will be written as raw bytes.

Utilities

Functions for mapping fpga_result values to meaningful error strings are provided by the utilities API.

utils.h

Utility functions and macros for the FPGA API.

Functions

const char *fpgaErrStr(fpga_result e)

Return human-readable error message

Returns a pointer to a human-readable error message corresponding to the provided fpga_error error code.

Return
Pointer to a descriptive error message string
Parameters
  • e: Error code (as returned by another FPGA API function

Samples

Code samples demonstrate how to use OPAE C API.

hello_fpga.c

A code sample illustrates the basic usage of the OPAE C API.

The sample is a host application that demonstrates the basic steps of interacting with FPGA using the OPAE library. These steps include:

  • FPGA enumeration
  • Resource acquiring and releasing
  • Managing shared memory buffer
  • MMIO read and write

The sample also demonstrates OPAE’s object model, such as tokens, handles, and properties.

The sample requires a native loopback mode (NLB) test image to be loaded on the FPGA. Refer to Quick Start Guide for full instructions on building, configuring, and running this code sample.

Defines

TEST_TIMEOUT
CL(x)
LOG2_CL
MB(x)
CACHELINE_ALIGNED_ADDR(p)
LPBK1_BUFFER_SIZE
LPBK1_BUFFER_ALLOCATION_SIZE
LPBK1_DSM_SIZE
CSR_SRC_ADDR
CSR_DST_ADDR
CSR_CTL
CSR_CFG
CSR_NUM_LINES
DSM_STATUS_TEST_COMPLETE
CSR_AFU_DSM_BASEL
CSR_AFU_DSM_BASEH
NLB0_AFUID
ON_ERR_GOTO(res, label, desc)
GETOPT_STRING

Functions

int usleep(unsigned)
void print_err(const char *s, fpga_result res)
fpga_result parse_args(int argc, char *argv[])
fpga_result find_fpga(fpga_guid afu_guid, fpga_token *accelerator_token, uint32_t *num_matches_accelerators)
fpga_result get_bus(fpga_token tok, uint8_t *bus)
int main(int argc, char *argv[])

Variables

struct config config = { .target = { .bus = -1, }, .open_flags = 0 }
struct config

Public Members

struct config::target target
int open_flags
struct target

Public Members

int bus

hello_events.c

A code sample of using OPAE event API.

This sample starts two processes. One process injects an artificial fatal error to sysfs; while the other tries to asynchronously capture and handle the event. This sample code exercises all major functions of the event API, including creating and destroying event handles, register and unregister events, polling on event file descriptor, and getting the OS object associated with an event. For a full discussion of OPAE event API, refer to event.h.

Defines

FME_SYSFS_INJECT_ERROR
ON_ERR_GOTO(res, label, desc)
GETOPT_STRING

Functions

int usleep(unsigned)
void print_err(const char *s, fpga_result res)
fpga_result inject_ras_fatal_error(fpga_token fme_token, uint8_t err)
void *error_thread(void *arg)
fpga_result parse_args(int argc, char *argv[])
fpga_result find_fpga(fpga_token *fpga, uint32_t *num_matches)
fpga_result get_bus(fpga_token tok, uint8_t *bus)
int main(int argc, char *argv[])

Variables

struct events_config events_config = { .target = { .bus = -1 } }
struct ras_inject_error

Public Members

union ras_inject_error::@1 ras_inject_error::@2
struct events_config

Public Members

struct events_config::target target
struct target

Public Members

int bus
union

Public Members

struct

Public Members