Metadata-Version: 2.1
Name: gpt4all-j
Version: 0.2.6
Summary: Python bindings for the C++ port of GPT4All-J model.
Home-page: https://github.com/marella/gpt4all-j
Author: Ravindra Marella
Author-email: mv.ravindra007@gmail.com
License: MIT
Description: # [GPT4All-J](https://github.com/marella/gpt4all-j) [![PyPI](https://img.shields.io/pypi/v/gpt4all-j)](https://pypi.org/project/gpt4all-j/) [![tests](https://github.com/marella/gpt4all-j/actions/workflows/tests.yml/badge.svg)](https://github.com/marella/gpt4all-j/actions/workflows/tests.yml)
        
        Python bindings for the [C++ port][gptj.cpp] of GPT4All-J model.
        
        > Please migrate to [`ctransformers`](https://github.com/marella/ctransformers) library which supports more models and has more features.
        
        ## Installation
        
        ```sh
        pip install gpt4all-j
        ```
        
        Download the model from [here](https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin).
        
        ## Usage
        
        ```py
        from gpt4allj import Model
        
        model = Model('/path/to/ggml-gpt4all-j.bin')
        
        print(model.generate('AI is going to'))
        ```
        
        [Run in Google Colab](https://colab.research.google.com/drive/1bd38-i1Qlx6_MvJyCTJOy7t8eHSNnqAx)
        
        If you are getting `illegal instruction` error, try using `instructions='avx'` or `instructions='basic'`:
        
        ```py
        model = Model('/path/to/ggml-gpt4all-j.bin', instructions='avx')
        ```
        
        If it is running slow, try building the C++ library from source. [Learn more](https://github.com/marella/gpt4all-j#c-library)
        
        ### Parameters
        
        ```py
        model.generate(prompt,
                       seed=-1,
                       n_threads=-1,
                       n_predict=200,
                       top_k=40,
                       top_p=0.9,
                       temp=0.9,
                       repeat_penalty=1.0,
                       repeat_last_n=64,
                       n_batch=8,
                       reset=True,
                       callback=None)
        ```
        
        #### `reset`
        
        If `True`, context will be reset. To keep the previous context, use `reset=False`.
        
        ```py
        model.generate('Write code to sort numbers in Python.')
        model.generate('Rewrite the code in JavaScript.', reset=False)
        ```
        
        #### `callback`
        
        If a callback function is passed, it will be called once per each generated token. To stop generating more tokens, return `False` inside the callback function.
        
        ```py
        def callback(token):
            print(token)
        
        model.generate('AI is going to', callback=callback)
        ```
        
        ## LangChain
        
        [LangChain](https://python.langchain.com/) is a framework for developing applications powered by language models. A LangChain LLM object for the GPT4All-J model can be created using:
        
        ```py
        from gpt4allj.langchain import GPT4AllJ
        
        llm = GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin')
        
        print(llm('AI is going to'))
        ```
        
        If you are getting `illegal instruction` error, try using `instructions='avx'` or `instructions='basic'`:
        
        ```py
        llm = GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin', instructions='avx')
        ```
        
        It can be used with other LangChain modules:
        
        ```py
        from langchain import PromptTemplate, LLMChain
        
        template = """Question: {question}
        
        Answer:"""
        
        prompt = PromptTemplate(template=template, input_variables=['question'])
        
        llm_chain = LLMChain(prompt=prompt, llm=llm)
        
        print(llm_chain.run('What is AI?'))
        ```
        
        ### Parameters
        
        ```py
        llm = GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin',
                       seed=-1,
                       n_threads=-1,
                       n_predict=200,
                       top_k=40,
                       top_p=0.9,
                       temp=0.9,
                       repeat_penalty=1.0,
                       repeat_last_n=64,
                       n_batch=8,
                       reset=True)
        ```
        
        ## C++ Library
        
        To build the C++ library from source, please see [gptj.cpp][gptj.cpp]. Once you have built the shared libraries, you can use them as:
        
        ```py
        from gpt4allj import Model, load_library
        
        lib = load_library('/path/to/libgptj.so', '/path/to/libggml.so')
        
        model = Model('/path/to/ggml-gpt4all-j.bin', lib=lib)
        ```
        
        ## License
        
        [MIT](https://github.com/marella/gpt4all-j/blob/main/LICENSE)
        
        [gptj.cpp]: https://github.com/marella/gptj.cpp
        
Keywords: gpt4all-j gpt4all gpt-j ai llm cpp
Platform: UNKNOWN
Classifier: Development Status :: 1 - Planning
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Description-Content-Type: text/markdown
Provides-Extra: tests
