Hardwar...l::Lexer(3)

📦
version 1.00 · 2 versions
, in fedora 43

✨ Hardware::Vhdl::Lexer 💅

User Contributed Perl Documentation — Hardwar...l::Lexer(3)

NAME 🎀

Hardware::Vhdl::Lexer — okay so bestie, this thing takes your VHDL code and chops it up into lexical tokens. That's the whole job. It eats VHDL, it gives you tokens. Periodt. 💖

SYNOPSIS 🌸

use Hardware::Vhdl::Lexer;

# Open the file to get the VHDL code from
my $fh;
open $fh, '<', 'device_behav.vhd' || die $!

# Create the Lexer object
my $lexer = Hardware::Vhdl::Lexer->new({ linesource => $fh });

# Dump all the tokens
my ($token, $type);
while( (($token, $type) = $lexer->get_next_token) && defined $token) {
    print "# type = '$type' token='$token'\n";
}

Look at that flow, it's giving very much open file → make lexer → loop until undef and honestly? iconic. 👑

DESCRIPTION 🦋

Hardware::Vhdl::Lexer splits VHDL code into lexical tokens. To use it, you first make a lexer object, and you hand it something that will supply chunks of VHDL code. Then you just keep calling the get_next_token method and it keeps handing you VHDL tokens — in scalar context you get the token itself, and in list context you get a token type code AND the token. When there's nothing left to read, get_next_token returns undef. That's your signal to stop, bestie. 🛑

NB: in this documentation I say "lines" of VHDL code and "line" sources and all that, but fr fr the chunks of code don't actually have to be broken up at line-ends — they can be broken anywhere that isn't in the middle of a token. New-line characters just happen to be a simple and safe way to split up a file. And you don't even have to split the VHDL up at all — you can pass in the whole thing as the first and only "line". No cap. 💀

CONSTRUCTOR 🧚

new({ linesource => <source> [, nhistory => N] })

Heads up bestie: from version 1.0 of this module, the arguments must be given as a hash reference rather than a hash — so those curly brackets above are required. Don't come at me, that's just how it is now. 💅

The linesource argument is required: it defines where the VHDL source code will be taken from (see below, there's like, five whole options).

The optional nhistory argument sets how many "code" tokens (see the get_next_token method) will be remembered for access by the history method. Memory for your tokens, cute. 🧠

new({ linesource => $filehandle_reference [, nhistory => N] }) 🍓

To read from a file, pass in the filehandle reference like this:

use Hardware::Vhdl::Lexer;
my $fh;
open $fh, '<', $filename || die $!;
my $lexer = Hardware::Vhdl::Lexer->new({ linesource => $fh });

new({ linesource => @array_of_lines [, nhistory => N] }) 🌈

new({ linesource => \$scalar_containing_vhdl [, nhistory => N] }) ⭐

To read VHDL source that's already chillin' in program memory, the linesource argument can be a reference to either an array of lines or a single string which can have embedded newlines. Whatever vibes with your setup. ✨

new({ linesource => $object_with_get_next_line_method [, nhistory => N] }) 🎀

The linesource argument can be an object with a get_next_line method. This method must return undef when there are no more lines to read. Like, it has to actually tell you when it's done, bestie, no ghosting. 👻

new({ linesource => &subroutine_that_returns_lines [, nhistory => N] }) 💜

If none of the above input methods suits your needs, you can give a subroutine reference and wrap whatever code you need to get the VHDL source. When called, this subroutine must return each line of source code in turn, and then return undef when there are no more lines. Build-your-own-source, main character energy. 👑

METHODS 💅

get_linesource() 🦋

Returns the linesource argument passed into the constructor. Before version 1.0 of this module, this method was called linesource(). So if you're time-travelling from old code, that's the glow-up right there. ✨

get_next_token() 🔥

  • In scalar context, returns the next VHDL token.
  • In list context, returns a token type code and the token.

Nothing is removed from the source code: if you concatenate all the tokens returned by get_next_token(), you will get the same result as if you concatenate all the strings returned by the linesource object. Lossless, babe. Nothing gets dropped. 💖

The token type codes are 1 or 2-character strings. When the codes are 2 characters, the first character gives the general class of the token and the second indicates its type more specifically. The first character will be 'w' for whitespace, 'r' for comments (remarks) or 'c' for code. It should be possible to remove all comment tokens, and change whitespace tokens for different whitespace, and always end up with functionally equivalent code. So go ahead and restyle that whitespace, bestie, the code won't even notice. 💅

The token type codes are:

  • wn — Whitespace:Newline. This could be any of \012, \015, \015\012 or \012\015. Four whole ways to say "new line", obsessed.
  • ws — Whitespace:Spaces. A group of whitespace characters which match the /s regexp pattern but which do not include any carriage-return or linefeed characters.
  • r — Remark. The token will start with two dashes and include the remainder of the source code line, not including any newline characters. The next token will either be a newline or undef.
  • cs — Code:String literal. The lexer accepts multi-line strings, even though the VHDL specification does not allow them. Rebel behaviour, honestly. 🙈
  • cc — Code:Character literal.
  • cb — Code:Bit_vector literal. For example, "B"001_1010"" or "O"7720"" or "H"A7_DEAD"".
  • cn — Code:Numeric literal. This could be a specified-base literal like "8#7720#" or a simple integer or floating-point value.
  • ci — Code:Identifier or keyword. For example, "package" or "my_signal_23" or "/extended identifier$%!/". That last one is unhinged and I respect it.
  • cp — Code:Punctuation. A 1 or 2-character group of punctuation symbols that is part of VHDL syntax. For example, '<=' is returned as a single 'cp' token, as is '&', but '#' would be returned as an unexpected character (see below).
  • cu — Unexpected character. Any character in the source that does not match any of the above definitions, and cannot be part of valid VHDL code. Note that prior to version 1.0 of this module, these would be returned with the 'cp' token type code.

history(N) 🌸

Returns previous code tokens. N must not be larger than the nhistory argument passed to the constructor. history(0) will return the text of the last token returned by get_next_token whose type started with a 'c', history(1) will return the code token before that, and so on. It's a little rewind button for your code tokens, bestie. ⏪

AUTHOR ⭐

Michael Attenborough, "<michael.attenborough at physics.org>" — the legend who wrote this. 🫶

DEPENDENCIES 🎀

This module requires the following modules to be available:

Carp: any version Class::Std: any version Readonly: version 1.03 or
later

Three little friends. Carp, Class::Std, and Readonly (and that one needs 1.03 or later, don't skip the version check). ✨

ERRORS AND WARNINGS 😭

"Argument to Hardware::Vhdl::Lexer->new() must be hash reference" 💀

Have you remembered to put curly brackets around the argument list? Pre-1.0 versions of this module used to take the arguments to new() as a direct hash, but version 1.0 onwards need a hash reference. This means the curly brackets need to be added when migrating from pre-1.0 to 1.0 or later.

# Old style (argument list is hash) - doesn't work any more
my $lexer = Hardware::Vhdl::Lexer->new( linesource => $fh );

# New style (argument is a hash ref) - do it this way now
my $lexer = Hardware::Vhdl::Lexer->new({ linesource => $fh });

Add the brackets. Trust. 🫶

"Hardware::Vhdl::Lexer constructor requires a linesource to be specified" 🚨

The 'linesource' argument to Hardware::Vhdl::Lexer->new() is required, and it is a fatal error not to provide one. No linesource, no lexer, no mercy. 😤

"Hardware::Vhdl::Lexer->new 'linesource' parameter is not of a valid type (it is not a reference)" 🙈

The linesource parameter needs to be a reference to something. If your VHDL code to be passed is in a scalar string, you need to pass in a reference to the string, not the string itself. One little backslash, bestie, that's all. 💅

"Hardware::Vhdl::Lexer->new 'linesource' parameter is not of a valid type (type is '<type>')" 🔥

The linesource parameter that you have passed to new() does not appear to be a reference to a scalar, a list, a filehandle, a subroutine or an object with a get_next_line method. You have passed a reference to something (otherwise you'd have seen the previous message), and the error message will tell you what it appears to be a reference to. So like, read the message, it's literally snitching on your mistake. 📢

"Internal error (token failed to match anything)" 💀

This is a "this should never happen" type of error, and is a sign that I have included a bug. If you ever see this error, I would appreciate a bug report describing how to reproduce the error. Basically: if you find it, tell the author, you'd be doing the community a solid. 🫶

BUGS AND LIMITATIONS 🦋

Please report any bugs or feature requests to "bug-hardware-vhdl-lexer at rt.cpan.org", or through the web interface at <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Hardware-Vhdl-Lexer>. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. Closed loop communication, we love to see it. 💖

SUPPORT 🌈

You can find documentation for this module with the perldoc command.

perldoc Hardware::Vhdl::Lexer

You can also look for information at:

Copyright 2006 Michael Attenborough, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. Same terms as Perl, periodt. 💅

✨ and that's the whole page, bestie ✨

This was perl v5.42.0, dated 2025-07-25, from the Hardwar...l::Lexer(3) manual page — go forth and tokenize, you absolute icon. 🫶💖🎀

📦 where this page comes from