About
What is KnowLM
The KnowLM project
KnowLM is an open-source language model framework for knowledge enhancement,
designed to assist researchers and developers in better managing updates and inaccuracies in large models
by providing a flexible and customizable toolkit and releasing corresponding models.
The technology, capabilities and content of KnowLM are as follows:
- Knowledge Prompting
- Knowledge Editing
- Knowledge Interaction
Knowledge Prompting.
KnowPrompt.
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction. WWW2022.
"Knowledge-informed Prompt Learning" seeks to enhance pre-trained models' understanding of specific domains and tasks by incorporating domain-related knowledge, such as knowledge graphs and text examples, through "Knowledge as Prompt".
This approach aims to unlock the full potential of pre-trained models and improve their performance significantly.
For More
Knowledge Editing.
EasyEdit.
Editing Large Language Models: Problems, Methods, and Opportunities. EMNLP2023.
Model Editing, aims to adjust an initial base model's behavior on the particular edit descriptor efficiently without influencing the model behavior on other samples.
Model Editing has strong connections with following topics:updating and fixing bugs for large language models, locating knowledge in large language models and lifelong learning, unlearning.
For More
Knowledge Interaction.
KnowAgent.
KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents.
Knowledge Interaction refers to the dynamic knowledge exchange and feedback between models and humans, environment, tools, or other models.
It is an important process to realize complex planning, agent intelligence, and large language model embodied cognition.
For More
Technologies for training LLMs
The full-scale pre-training code (providing conversion, construction, and loading of large corpora) and LoRA instruction fine-tuning code are open-sourced (support multi-machine multi-GPU). Provide examples for knowledge prompt, editing and interaction (still developing).
GithubResources
KnowLM Zoo
KnowLM Model Zoo
- We have currently open-sourced the base model of ZhiXi-13B and the extraction model of ZhiXi-13B, including the upcoming release of the base model KnowLM-7B based on LLaMA2.
Portfolio
What we've done
Paper Lists
Related works regarding knowledge prompt, editing and interaction.
- All
KnowPrompt
Knowledge-aware prompt-tuning approach with synergistic optimization for relation extraction.
EasyEdit
An easy-to-use framework to edit LLMs including GPT-J, Llama1/2, GPT-NEO, GPT2, T5.
KnowAgent
Knowledge-Augmented Planning for LLM-Based Agents.
RetroPrompt
Constructs an open-book knowledge-store from training instances and implements a retrieval mechanism for prompt learning.
RAP
A model-agnostic approach that utilizes schema and knowledge from human-annotated and weakly-supervised data as prompts.
TRICE
A two stage end-to-end framework that enables the model to continually learn through feedback derived from tool execution.
Acknowledgments
Our Sincere Thanks
Our sincerest thanks are extended to Langchain, Transformers, FastEdit, ROME, GRACE, MELO , PMET , and Self-Instruct for their invaluable contributions to our project. We have integrated parts of their source code into our work, and for this, we are deeply appreciative.Furthermore, we are grateful for the ongoing support and collaboration from our community. Special recognition goes to those who have diligently reported issues and shared their technical expertise. Your collective efforts have been instrumental in our project's success. 🙌