AI AND MACHINE LEARNING · 24 entries
AI and Machine Learning.
Entries on computational methods applied to research: machine learning, natural language processing, computer vision, language models, and contemporary neural architectures.
Algorithmic fairness ML subfield studying bias and discrimination in algorithmic systems, with formal criteria (demographic parity, equal opportunity, calibration) often in mathematical tension with each other. Barocas, Hardt, and Narayanan (2019) consolidated the reference.
AI/ML AUC-ROC Area Under the Receiver Operating Characteristic curve — discrimination metric for binary classifiers integrating performance across all decision thresholds. Hanley and McNeil (1982) formalized the probabilistic interpretation. Ranges from 0.5 (random) to 1.0 (perfect).
AI/ML BERT Pre-trained language model based on the Transformer architecture, developed by Google in 2018. Trained by *masked language modeling*, BERT established the pre-training + fine-tuning paradigm that dominated natural language processing until the generative LLM era.
AI/ML BERTopic Modern topic modeling algorithm combining contextual embeddings (BERT, Sentence-Transformers), dimensionality reduction (UMAP), clustering (HDBSCAN), and c-TF-IDF. Grootendorst (2022) consolidated. Often surpasses LDA in semantic coherence on small and medium corpora.
AI/ML Classification metrics Family of metrics for evaluating supervised classification models: accuracy, precision, recall, F1-score, AUC-ROC. Each captures a different aspect of the trade-off between false positives and false negatives. Powers (2011) synthesized the canonical framework.
AI/ML CLIP (Contrastive Language-Image Pre-training) Multimodal model pretrained by OpenAI that learns aligned image-text representations via contrastive learning on 400M web image-caption pairs. Radford et al. (2021). Enables zero-shot classification, image search, foundation for visual generative models.
AI/ML Cross-validation Predictive model evaluation technique that partitions the dataset into k subsets, trains k times alternating which subset serves as validation, and reports the mean error. Standard for small datasets where a fixed train/test split is unstable.
AI/ML Embeddings Dense vector representations of tokens — words, sentences, documents, images — in continuous fixed-dimensional space. Formalized in NLP by word2vec (Mikolov, 2013); today the foundation of semantic search, RAG, and most practical AI applications with text.
AI/ML Feature engineering Set of practices for transforming raw data into informative features for ML models: encoding, normalization, derived feature creation, selection, reduction. Domingos (2012) articulated it as a central variable of practical ML performance.
AI/ML Fine-tuning Adaptation of a pre-trained model to a specific task or domain via additional training over smaller labeled data. The dominant paradigm in NLP between 2018 and 2022, still relevant for BERT and specialized variants in technical domains.
AI/ML Fine-tuning vs prompt engineering Applied comparison between two paradigms for adapting LLMs: fine-tuning (weight update with specific data) and prompt engineering (instruction design without modifying the model). Trade-off among cost, control, latency, and generalization.
AI/ML Human annotation and inter-annotator agreement Manual labeling of data (text, image, audio) by human annotators, basis of supervised datasets in ML. Inter-annotator agreement (IAA) is measured via Cohen's kappa (1960), Krippendorff's alpha. Essential quality criterion.
AI/ML LLM (Large Language Model) Language model with billions to trillions of parameters, trained on massive text corpora via the Transformer architecture. Immediate ancestors: BERT (2018) and GPT-2 (2019). Milestones: GPT-3 (2020), instruction-tuned models (2022), multimodal models (2023+).
AI/ML Natural language processing (NLP) Field of artificial intelligence and computational linguistics dedicated to representing, processing, and generating human language with computational systems. Spans from classical syntactic analysis to large-scale language models like BERT and GPT.
AI/ML Overfitting Phenomenon in which a machine learning model fits the training-set sampling noise excessively, losing generalization ability. Detected by the gap between training error (low) and test error (high). Underfitting is the opposite problem.
AI/ML RAG (Retrieval-Augmented Generation) Retrieval-Augmented Generation: an architecture combining retrieval over an external document base with a language generation model. The current standard for answering questions with documentary grounding and reducing hallucination in LLMs.
AI/ML Semantic and instance segmentation Computer vision tasks that classify each pixel of an image. Semantic segmentation assigns a class label per pixel (without distinguishing instances); instance segmentation distinguishes individual objects of the same class. mIoU is the standard metric.
AI/ML Sentiment analysis NLP subfield that classifies affective polarity (positive, negative, neutral) or identifies specific emotions in text. Approaches evolved from manual lexicons to supervised classifiers to transformer-based models. Pang and Lee (2008) consolidated the field.
AI/ML SHAP values SHapley Additive exPlanations: ML model interpretability framework that attributes each feature's contribution to an individual prediction via Shapley values from cooperative game theory. Lundberg and Lee (2017) unified prior methods.
AI/ML Topic modeling (LDA) Latent Dirichlet Allocation: probabilistic generative model that discovers latent topics in a document corpus. Each document is a mixture of topics; each topic is a distribution over words. Blei, Ng, and Jordan (2003) consolidated the canonical classical NLP framework.
AI/ML Train/validation/test split Partitioning of a dataset into three disjoint subsets for machine learning: training (parameter fitting), validation (hyperparameter selection), and test (unbiased final evaluation). Methodological standard to avoid contamination.
AI/ML Transfer learning ML paradigm in which knowledge learned on a source task is transferred to a related target task, reducing labeled data and training time required. Pan and Yang (2010) consolidated the taxonomy. Foundation of pretrained-model use in modern deep learning.
AI/ML Transformer architecture Neural network architecture based exclusively on attention mechanisms, proposed by Vaswani et al. in 2017. Replaced recurrent networks in nearly every NLP task and became the structural foundation of BERT, GPT, Claude, Gemini, and the current generation of language models.
AI/ML YOLO (You Only Look Once) Family of real-time object detection models that reformulated detection as direct regression of bounding boxes and classes in a single network pass. Redmon et al. (2016). IoU is the central metric; mAP evaluates global performance.
AI/ML