Enum Token 

pub enum Token<'a> {
    Ident(Cow<'a, str>),
    Function(Cow<'a, str>),
    Url(Cow<'a, str>),
    Number(f32),
    String(Cow<'a, str>),
    Dimension(f32, Cow<'a, str>),
    Delim(char),
    Hash(Cow<'a, str>),
    AtKeyword(Cow<'a, str>),
    Whitespace,
    Comment(Cow<'a, str>),
    EOF,
}

CSS token produced by the tokenizer.

This represents syntactic units only. No semantic interpretation (length, color, etc.) is performed here.

String payloads are Cows borrowing from the input where no escape sequence was encountered, so common identifiers, units and comments avoid heap allocation entirely.

Variants

Ident(Cow<'a, str>)

Identifier token (e.g. div, color, --custom)

Function(Cow<'a, str>)

Function token (e.g. calc, var)

Url(Cow<'a, str>)

Unquoted url(...) token (e.g. url(data:image/png;base64,...)). The raw inner content is captured as a single lexeme, including characters that would otherwise be tokenized further (such as : or % in a base64 data: URL).

Number(f32)

Plain number without unit (e.g. 0, 1.5)

String(Cow<'a, str>)

Quoted string token (e.g. "hello", 'world')

Dimension(f32, Cow<'a, str>)

Dimension token (e.g. 10px, 50%, 2em)

Percentages are also represented as a dimension with % as the unit.

Delim(char)

Delimiter token (single-character symbols such as :, ;, >, +)

Hash(Cow<'a, str>)

Hash with String (e.g. #fff)

AtKeyword(Cow<'a, str>)

AtKeyword (e.g. @media)

Whitespace

One or more whitespace characters

Comment(Cow<'a, str>)

Comment

EOF

End-of-input marker

Trait Implementations

impl<'a> Clone for Token<'a>

fn clone(&self) -> Token<'a>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Debug for Token<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a, 'b> PartialEq<Token<'b>> for Token<'a>

Equality across token lifetimes so tokens borrowing from a live input can be compared against 'static literals (as in tests) or vice versa.

fn eq(&self, other: &Token<'b>) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.