Skip to main content

Intel

Optimum Intel is the interface between the 🤗 Transformers and Diffusers libraries and the different tools and libraries provided by Intel to accelerate end-to-end pipelines on Intel architectures.

Intel® Extension for Transformers (ITREX) is an innovative toolkit designed to accelerate GenAI/LLM everywhere with the optimal performance of Transformer-based models on various Intel platforms, including Intel Gaudi2, Intel CPU, and Intel GPU.

This page covers how to use optimum-intel and ITREX with LangChain.

Optimum-intel​

All functionality related to the optimum-intel and IPEX.

Installation​

Install using optimum-intel and ipex using:

pip install optimum[neural-compressor]
pip install intel_extension_for_pytorch

Please follow the installation instructions as specified below:

  • Install optimum-intel as shown here.
  • Install IPEX as shown here.

Embedding Models​

See a usage example. We also offer a full tutorial notebook "rag_with_quantized_embeddings.ipynb" for using the embedder in a RAG pipeline in the cookbook dir.

from langchain_community.embeddings import QuantizedBiEncoderEmbeddings

Intel® Extension for Transformers (ITREX)​

All functionality related to the intel-extension-for-transformers.

Installation​

Install intel-extension-for-transformers. For system requirements and other installation tips, please refer to Installation Guide

pip install intel-extension-for-transformers

Install other required packages.

pip install -U torch onnx accelerate datasets

Embedding Models​

See a usage example.

from langchain_community.embeddings import QuantizedBgeEmbeddings

Help us out by providing feedback on this documentation page: