Add fluxc compiler skeleton: token definitions and lexer
Introduces the fluxc Rust crate with the first two compiler stages: - token.rs: define_tokens! macro generates TokenKind enum and its Display impl from a single table covering all Flux tokens (literals, keywords, operators, punctuation, Eof/Unknown). Span (half-open u32 byte range) and Token<'src> (kind + span + zero-copy text slice) round out the module. - lexer.rs: Lexer<'src> produces Token<'src> from a source &str. Skips whitespace, // line comments, and /* */ block comments. Handles all integer bases (decimal, hex, octal, binary with _ separators), floats (fractional + exponent), string/char literals with escape sequences, and Unicode identifiers via unicode-xid. Implements Iterator<Item = Token> and includes 17 unit tests. Also adds .gitignore (ignores fluxc/target) and expands examples/fibonacci.flx with an iterative variant.
This commit is contained in:
16
fluxc/Cargo.lock
generated
Normal file
16
fluxc/Cargo.lock
generated
Normal file
@@ -0,0 +1,16 @@
|
||||
# This file is automatically @generated by Cargo.
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
|
||||
[[package]]
|
||||
name = "fluxc"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"unicode-xid",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "unicode-xid"
|
||||
version = "0.2.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
|
||||
Reference in New Issue
Block a user