GoKawiil - Latest Tech News & Aggregated Headlines

TikTok has turned culture into a feedback loop of impulse and machine learning

news.ycombinator.com Unknown 2025-09-30 16:08:39

Photo by Solen Feyissa on Unsplash As of September 2025, approximately 170 million Americans spend, on average, one hour every day in an app that is designed to maximize psychological grip. While Congress fixates on TikTok’s data collection usages, what hasn’t received enough attention is how the platform has successfully industrialized human attention itself. Where earlier media relied on polished narratives (films with arcs, shows with seasons), TikTok turned culture into a never-ending feedb

Topics: algorithm attention micro tiktok videos

Shop Amazon

TikTok won. Now everything is 60 seconds

news.ycombinator.com Unknown 2025-09-30 21:08:39

Photo by Solen Feyissa on Unsplash As of September 2025, approximately 170 million Americans spend, on average, one hour every day in an app that is designed to maximize psychological grip. While Congress fixates on TikTok’s data collection usages, what hasn’t received enough attention is how the platform has successfully industrialized human attention itself. Where earlier media relied on polished narratives (films with arcs, shows with seasons), TikTok turned culture into a never-ending feedb

Topics: algorithm attention micro tiktok videos

Shop Amazon

Almost anything you give sustained attention to will begin to loop on itself

news.ycombinator.com Henrik Karlsson 2025-10-10 13:29:49

Brioches and Knife, Eliot Hodgkin, 08/1961 1. When people talk about the value of paying attention and slowing down, they often make it sound prudish and monk-like. Attention is something we “have to protect.” And we have to “pay” attention—like a tribute. But we shouldn’t forget how interesting and overpoweringly pleasurable sustained attention can be. Slowing down makes reality vivid, strange, and hot. Let me start with the most obvious example. As anyone who has had good sex knows, susta

Topics: attention dopamine like saw things

Shop Amazon

From multi-head to latent attention: The evolution of attention mechanisms

news.ycombinator.com Unknown 2025-10-16 22:45:24

From Multi-Head to Latent Attention: The Evolution of Attention Mechanisms Vinithavn 7 min read · 15 hours ago 15 hours ago -- Listen Share Press enter or click to view image in full size What is attention? In any autoregressive model, the prediction of the future tokens is based on some preceding context. However, not all the tokens within this context equally contribute to the prediction, because some tokens might be more relevant than others. The attention mechanism addresses this by allow

Topics: attention context key query vectors

Shop Amazon

From Multi-Head to Latent Attention: The Evolution of Attention Mechanisms

news.ycombinator.com Unknown 2025-10-17 08:45:24

From Multi-Head to Latent Attention: The Evolution of Attention Mechanisms Vinithavn 7 min read · 15 hours ago 15 hours ago -- Listen Share Press enter or click to view image in full size What is attention? In any autoregressive model, the prediction of the future tokens is based on some preceding context. However, not all the tokens within this context equally contribute to the prediction, because some tokens might be more relevant than others. The attention mechanism addresses this by allow

Topics: attention context key query vectors

Shop Amazon

Scientists Can’t Figure Out Why Just Walking In Nature Appears to Quickly Heal Your Brain Rot

futurism.com Unknown 2025-11-02 23:16:21

Image by Getty / Futurism Mental Health "Go outside" or "touch grass" are common rejoinders deployed in online arguments these days. And, at least for those of us whose brains have probably melted from spending too much time on an app where said arguments take place, it turns out it's pretty sound advice. As the New York Times reports, there's a growing body of evidence suggesting that simply spending time in nature can instantly boost your algorithm-addled brain's attention span. It's part of

Topics: attention nature nyt theory university

Shop Amazon

Attention Is the New Big-O: A Systems Design Approach to Prompt Engineering

news.ycombinator.com Alex Chesser 2025-11-06 21:29:58

1. Understanding Attention: Your First Step to Better Prompts If you’re human, you’re probably reading this from left to right. You might not have stopped for a moment to consider the fact that your LLM doesn’t read in the same order as you or I. Instead, it weights relationships between all tokens at once, with position and clustering dramatically changing what gets noticed. In working with an LLM the structure you choose can have a greater impact on your results than the precise words you ch

Topics: attention model prompt prompts structure

Shop Amazon

How attention sinks keep language models stable

news.ycombinator.com Guangxuan Xiao 2025-11-24 04:53:10

We discovered why language models catastrophically fail on long conversations: when old tokens are removed to save memory, models produce complete gibberish. We found models dump massive attention onto the first few tokens as "attention sinks"—places to park unused attention since softmax requires weights to sum to 1. Our solution, StreamingLLM, simply keeps these first 4 tokens permanently while sliding the window for everything else, enabling stable processing of 4 million+ tokens instead of j

Topics: attention models sink sinks tokens

Shop Amazon

How Attention Sinks Keep Language Models Stable

news.ycombinator.com Guangxuan Xiao 2025-11-26 10:53:10

We discovered why language models catastrophically fail on long conversations: when old tokens are removed to save memory, models produce complete gibberish. We found models dump massive attention onto the first few tokens as "attention sinks"—places to park unused attention since softmax requires weights to sum to 1. Our solution, StreamingLLM, simply keeps these first 4 tokens permanently while sliding the window for everything else, enabling stable processing of 4 million+ tokens instead of j

Topics: attention models sink sinks tokens

Shop Amazon

LLM architecture comparison

news.ycombinator.com Sebastian Raschka 2026-01-03 06:56:01

It has been seven years since the original GPT architecture was developed. At first glance, looking back at GPT-2 (2019) and forward to DeepSeek-V3 and Llama 4 (2024-2025), one might be surprised at how structurally similar these models still are. Sure, positional embeddings have evolved from absolute to rotational (RoPE), Multi-Head Attention has largely given way to Grouped-Query Attention, and the more efficient SwiGLU has replaced activation functions like GELU. But beneath these minor refi

Topics: attention deepseek figure gemma model

Shop Amazon

The Big LLM Architecture Comparison

news.ycombinator.com Sebastian Raschka 2026-01-03 12:56:01

It has been seven years since the original GPT architecture was developed. At first glance, looking back at GPT-2 (2019) and forward to DeepSeek-V3 and Llama 4 (2024-2025), one might be surprised at how structurally similar these models still are. Sure, positional embeddings have evolved from absolute to rotational (RoPE), Multi-Head Attention has largely given way to Grouped-Query Attention, and the more efficient SwiGLU has replaced activation functions like GELU. But beneath these minor refi

Topics: attention deepseek figure gemma model

Shop Amazon

The Tradeoffs of SSMs and Transformers

news.ycombinator.com Goomba Ai Lab 2026-01-26 06:12:06

This blog post was adapted from a talk I’ve given a handful of times over the last year. It was meant to be a high-level talk accessible to a fairly broad audience, but hopefully has some interesting insights, opinions, and intuitions around sequence models for the dedicated researchers too. State Space Models Just so we’re on the same page, I’ll start by defining what I mean by a state space model. (This section isn’t strictly necessary to get to the main part of this post though; feel free t

Topics: attention data models state transformers

Shop Amazon

VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

news.ycombinator.com Woosuk Kwon 2026-01-28 17:16:20

GitHub | Documentation | Paper LLMs promise to fundamentally change how we use AI across all industries. However, actually serving these models is challenging and can be surprisingly slow even on expensive hardware. Today we are excited to introduce vLLM, an open-source library for fast LLM inference and serving. vLLM utilizes PagedAttention, our new attention algorithm that effectively manages attention keys and values. vLLM equipped with PagedAttention redefines the new state of the art in LL

Topics: llm memory pagedattention throughput vllm

Shop Amazon

I have reimplemented Stable Diffusion 3.5 from scratch in pure PyTorch

news.ycombinator.com Unknown 2026-03-11 17:56:46

miniDiffusion miniDiffusion is a reimplementation of the Stable Diffusion 3.5 model in pure PyTorch with minimal dependencies. It's designed for educational, experimenting, and hacking purposes. It's made with the mindset of having the least amount of code necessary to recreate Stable Diffusion 3.5 from scratch, with only ~2800 spanning from VAE to DiT to the Train and Dataset scripts. -Files: The main Stable Diffusion model code is located in dit.py, dit_components.py, and attention.py. The d

Topics: attention diffusion dit model py

Shop Amazon

DeepDive in everything of Llama3: revealing detailed insights and implementation

news.ycombinator.com Unknown 2025-11-09 08:57:13

[ View in English | 中文版文档点这里 ] This project is an enhanced version based on naklecha/llama3-from-scratch. It has been comprehensively improved and optimized on the basis of the original project, aiming to help everyone more easily understand and master the implementation principle and the detailed reasoning process of the Llama3 model. Thanks to the contributions of the original author :) The following are the core improvements of this project: Structural Optimization The presentation se

Topics: attention inf token tokens torch

Shop Amazon

Latest Tech News

TikTok has turned culture into a feedback loop of impulse and machine learning

TikTok won. Now everything is 60 seconds

Almost anything you give sustained attention to will begin to loop on itself

From multi-head to latent attention: The evolution of attention mechanisms

From Multi-Head to Latent Attention: The Evolution of Attention Mechanisms

Scientists Can’t Figure Out Why Just Walking In Nature Appears to Quickly Heal Your Brain Rot

Attention Is the New Big-O: A Systems Design Approach to Prompt Engineering

How attention sinks keep language models stable

How Attention Sinks Keep Language Models Stable

LLM architecture comparison

The Big LLM Architecture Comparison

The Tradeoffs of SSMs and Transformers

VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

I have reimplemented Stable Diffusion 3.5 from scratch in pure PyTorch

DeepDive in everything of Llama3: revealing detailed insights and implementation

About GoKawiil

Privacy

Advertising

Latest Tech News

Trending Topics

Hot Now

Popular

Emerging

About GoKawiil

Privacy

Advertising