I'm trying to fine-tune a GPT-2 model using the Hugging Face Transformers library, but I'm encountering a segmentation fault during training. Here's a minimal reproducible example: import torch from transformers import GPT2Tokenizer, GPT2LMHeadModel, Trainer, TrainingArguments, DataCollatorForLanguageModeling from datasets import load_dataset…
I have finetuned a gemma 7b LLM from HuggingFace using Lora and stored the model as a compressed .tar.gz file. I have finetuned locally in sagemaker. this is my .tar.gz file structure of finetuned model : finetuned_gemma/model-00004-of-00004.safetensors finetuned_gemma/tokenizer_config.json finetuned_gemma/model.safetensors.index.json finetuned_gemma/config.json…
OS: Ubuntu 18.04 LTS CUDA: 11.3 GPU: NVIDIA P5000 Quadro IDE: Jupyter Notebook Environment: VirtualEnv (venv) Code: # Importing the required libraries import torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig # Defining the name of the Falcon model model_name =…
So I have a JSON dataset that is in the following format: [{ "answer": "...", "question": "...", "context": "..." }, ... ] All of the fields are normal plaintext. My goal is train a pretrained BERT model on it for…
I'm working on a fastAPI project on an Amazon EC2 instance running Ubuntu 20.04.5. The nature of the project requires me to have several custom types (written by me) and third party types (from HuggingFace transformers, hf peft, Langchain) as…
I have created a API script with FastAPI to run a HF transformers' NER model. However I am puzzled to see that the output returning from the API doesn't match the output I get if run the model directly (the…
I am running the pytorch code below. I'm running the code in a jupyter notebook. the noebook is running on my ubuntu server. I'm trying to download the llama2-70b-chat model from hugging face. my goal is to download the model…
I am having the data as docx files and I want to use them to fine-tune the Falcon model. From what I see the data used to train the model was in json format. How can I convert my data…
I have been unable to get any output from the model even waiting 10 minutes. Running on a Azure Notebook with a Compute instance Standard_E4ds_v4, 4 core, 32GB. Any assistance is appreciated. Code: !source activate llm_env %pip install conda import…
I've followed this tutorial (colab notebook) in order to finetune my model. Trying to load my locally saved model model = AutoModelForCausalLM.from_pretrained("finetuned_model") yields Killed. Trying to load model from hub: yields import torch from peft import PeftModel, PeftConfig from transformers…