NLP - Neural Language Models

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled

NLP - Neural Language Models

Word embeddings, skip-gram Word2Vec with negative sampling and cosine similarity of words

  • 0 Rating
  • 0 Reviews
  • 2 Students Enrolled
  • Wishlist
  • Free
  • Course Includes
  • PDF slides, 18 pages
  • LaTeX source and figures (zip)
  • Quiz, 4 questions
Tags:
Word Embeddings Word2Vec Neural Language Models Semantic Similarity NLP



What you will learn

  • Represent words as dense vectors of 100 to 300 dimensions and measure their similarity with cosine distance
  • Train a skip-gram model with two embedding matrices and negative sampling
  • Evaluate embeddings on analogy questions and name the limits of one vector per word

Courselet Content

2 components
NLP - Neural Language Models (pdf)
301.68 K

Requirements

  • N-gram language models, vectors and dot products, basic PyTorch

General Overview

Description

Words that occur in similar contexts have similar meanings, and the vectors of such words lie close together. A one-hot ID over a 50,000-word vocabulary has 50,000 entries, a dense embedding 100 to 300. In skip-gram Word2Vec, context words are predicted from a center word, with two embedding matrices and negative sampling. Similarity is measured by cosine distance, and vector arithmetic such as king - man + woman gives queen. Embeddings are evaluated on semantic and syntactic analogy questions. A single vector per word is an average over the senses of bank, has no word order and is tied to a fixed vocabulary.

Courses that include this CL

blog
Last Updated 19th November 2026
  • 5
  • Free

Recommended for you

blog
Last Updated 15th January 2025
  • 9
blog
Last Updated 10th December 2023
  • 5
blog
Last Updated 19th July 2023
  • 0
  • 0
blog
Last Updated 16th June 2023
  • 7
blog
Last Updated 5th June 2025
  • 3
  • Free
blog
Last Updated 16th January 2023
  • 49
  • Free
blog
Last Updated 7th January 2023
  • 8
  • Free
blog
Last Updated 14th March 2025
  • 52
  • Free
blog
Last Updated 20th August 2025
  • 50
  • Free
blog
Last Updated 19th November 2023
  • 7
blog
Last Updated 7th November 2022
  • 16
  • Free
blog
Last Updated 21st March 2025
  • 251
  • Free

Meet the instructors !

instructor
About the Instructor

https://www.joergosterrieder.com/