Erasure Code API library written in C with pluggable Erasure Code backends.
Go to file
Tim Burke e1377f6f50 CI: Remove centos-7 jobs
CentOS 7 will go EOL later this year, and infra wants to drop the nodes
soon-ish -- don't make them wait on our account.

Change-Id: Iaa31fc95ed2d100726adf60807c8bf95599afd3f
2024-03-01 10:39:38 -08:00
doc re-org of README 2017-06-07 10:59:01 -04:00
include Use spaces for code indent, instead of tabs 2024-01-27 17:03:16 +09:00
playbooks Add CentOS 9 Stream job 2022-08-30 12:12:08 -07:00
roles Switch from yasm to nasm 2023-07-13 16:59:36 +00:00
src Use spaces for code indent, instead of tabs 2024-01-27 17:03:16 +09:00
test Fix bounds check in get_fragment_partition 2023-10-04 15:08:07 -07:00
.gitignore Remove remaining m4/ references 2015-06-21 22:17:08 +00:00
.gitreview Update .gitreview following project rename 2019-06-12 10:30:14 -07:00
.mailmap Release 1.6.2 2020-10-08 14:46:14 -07:00
.zuul.yaml CI: Remove centos-7 jobs 2024-03-01 10:39:38 -08:00
AUTHORS Release 1.6.2 2020-10-08 14:46:14 -07:00
COPYING Reformat license file 2020-03-10 18:38:10 -07:00
ChangeLog Release 1.6.3 2022-04-08 23:59:40 +00:00
Makefile.am Fix building with LD_LIBRARY_PATH set 2018-10-01 16:35:35 +02:00
NEWS Add NEWS and README back 2016-05-27 14:27:11 -04:00
README Add NEWS and README back 2016-05-27 14:27:11 -04:00
README.md Add missing dependency to README 2023-10-04 15:13:15 -07:00
Xorcode.pc.in autoconf: Add Xorcode pkgconfig, other minor fixes 2014-06-29 21:42:09 -07:00
autogen.sh Add -f argument to autoreconf to regenerate aclocal macros 2015-06-22 02:49:16 +00:00
bindep.txt Use zlib for CRC-32 2017-07-06 17:40:38 +00:00
configure.ac configure: use more idiomatic test command 2024-01-29 18:40:38 -05:00
erasurecode.pc.in Add dl to pkg-config metadata 2017-07-13 13:26:00 -07:00
erasurecode_rs_vand.pc.in Add pkg-config template for liberasurecode_rs_vand 2015-06-23 04:07:26 +00:00
get_flags_from_cpuid.c Fix get_supported_flags() arguments 2015-06-22 02:54:25 +00:00

README.md

liberasurecode

liberasurecode is an Erasure Code API library written in C with pluggable Erasure Code backends.


Highlights

  • Unified Erasure Coding interface for common storage workloads.

  • Pluggable Erasure Code backends - liberasurecode supports the following backends:

    • 'liberasurecode_rs_vand' - Native, software-only Erasure Coding implementation that supports a Reed-Solomon backend
    • 'Jerasure' - Erasure Coding library that supports Reed-Solomon, Cauchy backends [1]
    • 'ISA-L' - Intel Storage Acceleration Library - SIMD accelerated Erasure Coding backends [2]
    • 'SHSS' - NTT Lab Japan's hybrid Erasure Coding backend [4]
    • 'Flat XOR HD' - built-in to liberasurecode, based on [3]
    • 'libphazr' - Phazr.IO's erasure code backend with built-in privacy [5]
    • 'NULL' template backend implemented to help future backend writers
  • True 'plugin' architecture - liberasurecode uses Dynamically Loaded (DL) libraries to realize a true 'plugin' architecture. This also allows one to build liberasurecode indepdendent of the Erasure Code backend libraries.

  • Cross-platform - liberasurecode is known to work on Linux (Fedora/Debian flavors), Solaris, BSD and Darwin/Mac OS X.

  • Community support - Developed alongside Erasure Code authority Kevin Greenan, liberasurecode is an actively maintained open-source project with growing community involvement (Openstack Swift, Ceph, PyECLib, NTT Labs).


Active Users


Build and Install

Install dependencies -

Debian/Ubuntu hosts:

 $ sudo apt-get install build-essential autoconf automake libtool zlib1g-dev

Fedora/RedHat/CentOS hosts:

 $ sudo yum install -y gcc make autoconf automake libtool zlib-devel

To build the liberasurecode repository, perform the following from the top-level directory:

 $ ./autogen.sh
 $ ./configure
 $ make
 $ make test
 $ sudo make install

Getting Help


References

[1] Jerasure, C library that supports erasure coding in storage applications, http://jerasure.org

[2] Intel(R) Storage Acceleration Library (Open Source Version), https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version

[3] Greenan, Kevin M et al, "Flat XOR-based erasure codes in storage systems", https://web.archive.org/web/20161001210233/https://www.kaymgee.com/Kevin_Greenan/Publications_files/greenan-msst10.pdf

[4] Kota Tsuyuzaki tsuyuzaki.kota@lab.ntt.co.jp, "NTT SHSS Erasure Coding backend"

[5] Jim Cheung support@phazr.io, "Phazr.IO libphazr erasure code backend with built-in privacy"