Glossary

0-9

0-shot learning 1-shot learning 2-stage detector 3D convolution 4D data 5G + AI 6DoF pose estimation 7D representation 8-bit quantization 9-layer network

A

AGI / Artificial General Intelligence Algorithm Artificial Intelligence (AI)Attention Autoencoder

B

Backpropagation Batch Normalization BERT Bias Boosting

C

Chatbot Classifier / Classification Clustering CNN / Convolutional Neural Network Cross-Validation

D

Data Augmentation Deep Learning Deepfake Deterministic Model Discriminative Model

E

Embedding Encoder Ensemble Learning Epoch Explainable AI (XAI)

F

Feature Extraction Fine-tuning Forward Propagation Foundation Model Fusion / Multimodal Fusion

G

GAN / Generative Adversarial Network Generative AI Gradient Descent Graph Neural Network (GNN)Grounding

H

Hallucination Heuristic Hidden Layer Hierarchical Model Hyperparameter

I

Imbalanced Data Instance / Sample Instruction tuning Intelligence Amplification / Augmentation Interpretability

J

JAX Jittering Joint Embedding JSONL / JSON-lines Juxtaposition

K

K-means Clustering K-Shot Learning Kernel Trick KL Divergence (Kullback–Leibler Divergence)Knowledge Distillation

L

Large Language Model (LLM)Latent Variable Learning Rate Loss Function LSTM / Long Short-Term Memory

M

Machine Learning (ML)Meta-learning Model Multi-head Attention Multimodal / Multimodality

N

Neural Network NLP / Natural Language Processing NLU / Natural Language Understanding Normalization Novelty Detection / Anomaly Detection

O

Objective Function One-hot Encoding Online Learning Optimizer Overfitting

P

Parameter Policy / Reinforcement Learning Policy Pooling Pretraining Prompt

Q

Q-learning Quality Estimation Quantization Query Queue / Buffer

R

Regularization Reinforcement Learning (RL)Representation Learning Retrieval Augmented Generation (RAG)RNN / Recurrent Neural Network

S

Sampling Self-Supervised Learning Sequence Modeling Softmax Supervised Learning

T

Tokenizer Training Data Transfer Learning Transformer Tuning / Hyperparameter Tuning

U

U-Net Uncertainty Estimation Underfitting Universal Approximation Theorem Unsupervised Learning

V

Validation Set Vanishing / Exploding Gradient Variational Autoencoder (VAE)Vector Embedding Vision Transformer (ViT)

W

Weak Supervision Weight Decay Whitening / Whitening Transformation Word Embedding Workflow

X

X-axis / feature axis XAI / Explainable AI XLM XLNet XOR problem

Y

Y-axis / feature axis Y-transform / YUV YAGNI (You Aren't Gonna Need It)Yield (model yield / throughput)Yoga of AI

Z

Z-score Normalization Zero-centric / Zero-bias initialization Zero-gradient phenomenon Zero-shot Learning / Zero-shot inference Zygosity in augmentation

What is Fine-tuning

Fine-tuning is a crucial concept in machine learning and artificial intelligence, referring to the process of further training a pre-trained model to improve its performance on specific tasks. This technique allows researchers to adapt a model to particular datasets, enhancing its accuracy and effectiveness.

The background of fine-tuning can be traced back to the rapid development of deep learning, especially with the emergence of large-scale pre-trained models like BERT and GPT, which have shown outstanding performance across various tasks and facilitated the widespread adoption of fine-tuning.

Typically, fine-tuning involves selecting a pre-trained model, loading its weights, and then training it on a specific dataset. This approach enables researchers to achieve good results even with smaller datasets since the model has already learned useful features.

Fine-tuning is widely applied in scenarios such as natural language processing and computer vision. For instance, in sentiment analysis tasks, researchers can fine-tune a pre-trained language model to better understand the domain-specific terminology and context.

Looking ahead, fine-tuning may continue to evolve, particularly in the context of automation and unsupervised learning, with researchers exploring ways to enhance its efficiency and effectiveness. However, as model sizes grow, fine-tuning will face new challenges.

Advantages include saving time and resources while improving task-specific model performance. On the other hand, drawbacks may include the risk of overfitting and the need for a well-curated dataset for specific tasks.

It's essential to note that selecting an appropriate learning rate and number of training epochs is crucial when fine-tuning, as these factors directly impact the final model's performance.