Glossary

0-9

0-shot learning 1-shot learning 2-stage detector 3D convolution 4D data 5G + AI 6DoF pose estimation 7D representation 8-bit quantization 9-layer network

A

AGI / Artificial General Intelligence Algorithm Artificial Intelligence (AI)Attention Autoencoder

B

Backpropagation Batch Normalization BERT Bias Boosting

C

Chatbot Classifier / Classification Clustering CNN / Convolutional Neural Network Cross-Validation

D

Data Augmentation Deep Learning Deepfake Deterministic Model Discriminative Model

E

Embedding Encoder Ensemble Learning Epoch Explainable AI (XAI)

F

Feature Extraction Fine-tuning Forward Propagation Foundation Model Fusion / Multimodal Fusion

G

GAN / Generative Adversarial Network Generative AI Gradient Descent Graph Neural Network (GNN)Grounding

H

Hallucination Heuristic Hidden Layer Hierarchical Model Hyperparameter

I

Imbalanced Data Instance / Sample Instruction tuning Intelligence Amplification / Augmentation Interpretability

J

JAX Jittering Joint Embedding JSONL / JSON-lines Juxtaposition

K

K-means Clustering K-Shot Learning Kernel Trick KL Divergence (Kullback–Leibler Divergence)Knowledge Distillation

L

Large Language Model (LLM)Latent Variable Learning Rate Loss Function LSTM / Long Short-Term Memory

M

Machine Learning (ML)Meta-learning Model Multi-head Attention Multimodal / Multimodality

N

Neural Network NLP / Natural Language Processing NLU / Natural Language Understanding Normalization Novelty Detection / Anomaly Detection

O

Objective Function One-hot Encoding Online Learning Optimizer Overfitting

P

Parameter Policy / Reinforcement Learning Policy Pooling Pretraining Prompt

Q

Q-learning Quality Estimation Quantization Query Queue / Buffer

R

Regularization Reinforcement Learning (RL)Representation Learning Retrieval Augmented Generation (RAG)RNN / Recurrent Neural Network

S

Sampling Self-Supervised Learning Sequence Modeling Softmax Supervised Learning

T

Tokenizer Training Data Transfer Learning Transformer Tuning / Hyperparameter Tuning

U

U-Net Uncertainty Estimation Underfitting Universal Approximation Theorem Unsupervised Learning

V

Validation Set Vanishing / Exploding Gradient Variational Autoencoder (VAE)Vector Embedding Vision Transformer (ViT)

W

Weak Supervision Weight Decay Whitening / Whitening Transformation Word Embedding Workflow

X

X-axis / feature axis XAI / Explainable AI XLM XLNet XOR problem

Y

Y-axis / feature axis Y-transform / YUV YAGNI (You Aren't Gonna Need It)Yield (model yield / throughput)Yoga of AI

Z

Z-score Normalization Zero-centric / Zero-bias initialization Zero-gradient phenomenon Zero-shot Learning / Zero-shot inference Zygosity in augmentation

What is Forward Propagation

Forward Propagation is a fundamental concept in neural networks, essential for training and inference processes. It refers to the flow of signals from the input layer to the output layer within a neural network. During this process, input data is passed through neurons, weighted, and transformed by activation functions, culminating in an output result. Understanding this process is crucial for designing effective deep learning models.

The significance of Forward Propagation lies in its role as the foundation for calculating predictions from input data. It allows the neural network to generate output, which is vital for providing feedback during training. A solid grasp of how Forward Propagation works is key to building efficient neural networks.

During Forward Propagation, each layer's output becomes the input for the next layer. Each neuron computes the weighted sum of inputs and applies a non-linear activation function. This process can be efficiently executed using matrix operations, especially in large datasets and complex models.

Forward Propagation is commonly used in various applications, including image recognition, natural language processing, and recommendation systems. For instance, in image classification tasks, input images are processed through multiple convolutional and fully connected layers during Forward Propagation to yield the probability distribution of each class.

As deep learning continues to evolve, the efficiency and accuracy of Forward Propagation are being enhanced. Researchers are exploring more efficient computational methods and network architectures to enable faster processing of large-scale data.

The advantages of Forward Propagation include its intuitiveness and efficiency, allowing for rapid computation of predictions. However, its dependency on network architecture can lead to overfitting if the model is overly complex.

When designing neural networks, it is important to appropriately configure the number of neurons in each layer and the activation functions used to strike a balance between model performance and computational efficiency.

What is Forward Propagation - Glossary