About: The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction Goto Sponge NotDistinct Permalink

An Entity of Type : schema:ScholarlyArticle, within Data Space : covidontheweb.inria.fr associated with source document(s)

Attributes	Values
type	Academic Article research paper schema:ScholarlyArticle
isDefinedBy	Covid-on-the-Web dataset
title	The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction
Creator	Le Corff, Sylvain Martin, Alice Ollion, Charles Pietquin, Olivier Samovar, ∓ »more»
source	ArXiv
abstract	This paper introduces the Sequential Monte Carlo Transformer, an original approach that naturally captures the observations distribution in a recurrent architecture. The keys, queries, values and attention vectors of the network are considered as the unobserved stochastic states of its hidden structure. This generative model is such that at each time step the received observation is a random function of these past states in a given attention window. In this general state-space setting, we use Sequential Monte Carlo methods to approximate the posterior distributions of the states given the observations, and then to estimate the gradient of the log-likelihood. We thus propose a generative model providing a predictive distribution, instead of a single-point estimate.
has issue date	2020-07-15(xsd:dateTime)
has license	arxiv
sha1sum (hex)	08687b31e7eca10e98d3aafbbb12f4e8a5149e0b
resource representing a document's title	The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction
resource representing a document's body	covid:08687b31e7eca10e98d3aafbbb12f4e8a5149e0b#body_text
is schema:about of	named entity 'structure' named entity 'received' named entity 'sequence' named entity 'FUNCTION' named entity 'PAST' »more»

Faceted Search & Find service v1.13.91 as of Mar 24 2020

Alternative Linked Data Documents: Sponger | ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3229 as of Jul 10 2020, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (94 GB total memory)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2025 OpenLink Software