> ... and frankly I really like just plain vanilla Lua. I think it matches perfectly with Rust because they're so different, I think having an Rust embedded language that frees you from even having to think about ownership is very powerful because it can be used for things where having to think about ownership can be more trouble than its worth. Let each language play to their strengths, and Rust and Lua in a lot of ways have complementary strengths.
Tons of awesome technical details in the post and I've always felt that Lua's co-routines are incredibly undervalued. We used to run Lua in all sorts of crazy places and it really punches above its weight.
The above quote + zoom out at the end also makes some really good observations. Some of the most powerful systems I've worked on picked a couple core technologies that were complementary with good interop rather than a one-size fits all. Will be interesting to see where Piccolo goes.
10000truths 5 days ago [-]
> However, piccolo works very hard to guarantee that piccolo::Executor::step returns in a bounded amount of time, even without the cooperation of the running Lua code.
The issue is that there exist ways to make a single bytecode instruction take an unbounded amount of time to execute. For example, Lua's "string.find()" is implemented in native code. The interpreter only sees a single OP_CALL opcode, so it will count it as 1 instruction executed. But the actual execution time of the native implementation of string.find() is dependent on its inputs, which are not only variable length strings, but can be maliciously crafted to run in exponential time. Here's an example, shamelessly stolen from Mike Pall himself:
The only way to solve this is to track execution time within the native code as well (i.e. make them fuel-aware), and ensure that they abort immediately if they exhaust the fuel.
haileys 5 days ago [-]
> The only way to solve this is to track execution time within the native code as well (i.e. make them fuel-aware), and ensure that they abort immediately if they exhaust the fuel.
You will find that this is precisely the approach Piccolo takes.
wesamco 5 days ago [-]
I love how there are many Piccolo REPLs embedded in the article and interleaved with the paragraphs.
Piccolo looks amazing, I got a perfect use case for it, and I'm excited for when I get the chance to use it, thank you for working on it.
rnmmrnm 5 days ago [-]
A bit off topic: I just wrote a bunch of lua C debug api code (lua_sethook) to pre-emptively run and context-switch multiple lua "coroutines" (well not so "co").
Is this library offering a lua implementation more well-designed for this use-case? I got all this code to unload the coroutine stack to store and and reload it before continuing it later. Does having C bindings to this library makes sense?
interroboink 5 days ago [-]
I think yes?
My understanding is that the "stackless" concept here means that it does not store its execution state in the "C runtime stack" (or Rust, in this case).
So, there is some blob of memory describing that info, managed by Piccolo, rather than it residing on the "real" OS execution stack.
In particular, for call chains like: Lua -> C -> Lua -> C (or so), it is normally hard to save/restore the "C" parts of that chain, since you need to peek into the not-normally-accessible and platform-specific C runtime stack details. I wonder: how are you doing it in your system?
In Piccolo, I imagine it would be easier, since even the "-> C ->" portions of the stack are being managed by Piccolo, not the base C runtime. I don't know what the APIs to access that stuff actually look like, though.
Aside: have you looked at Pluto/Eris for Lua serialization of coroutines?
----
EDIT Yes, it seems like the section The "Big Lie" is right up your alley (:
rnmmrnm 5 days ago [-]
Well the only thing that's really itching me is the fact that the whole lua debug.* section is documented as
>You should exert care when using this library. Several of its functions violate basic assumptions about Lua code (e.g., that variables local to a function cannot be accessed from outside; that userdata metatables cannot be changed by Lua code; that Lua programs do not crash) and therefore can compromise otherwise secure code
(from the manual)
kinda creeps me out.
Dylan16807 5 days ago [-]
If you find a debugger that can't ruin a function's local variables then that debugger sucks.
rnmmrnm 5 days ago [-]
obviously, but i'm not so keen to turn it into a pre-emptive scheduler either lol.
rnmmrnm 5 days ago [-]
reading beyond the first paragraphs, I see this practically what he advocates to do :P
sanxiyn 5 days ago [-]
She, not he. kyren is a woman.
rnmmrnm 5 days ago [-]
Terribly sorry!
camgunz 5 days ago [-]
I really enjoyed the "Zooming Out" section, a lot of good food for thought in there.
klaussilveira 5 days ago [-]
I wonder if a Rust reimplementation of Lua will, one day, be faster than LuaJIT.
PhilipRoman 5 days ago [-]
Good point, it is well known that speed of machine code varies greatly depending on what language was used to generate it. Jokes aside, I really don't see a point in this. The safety guarantees of Rust are thrown out of the window the second you call dynamically generated code.
animaomnium 5 days ago [-]
Clearly that isn't the case, surely Mike Pall would have heard of it, when he brought back LuaJIT from the future, that is.
crushmuch 5 days ago [-]
[flagged]
__s 5 days ago [-]
Not without being a JIT. See Cranelift. I've used it to make a Befunge JIT in Rust
ufo 5 days ago [-]
Furthermore, LuaJIT's baseline interpreter is hand-crafted assembly and that's gonna be hard to beat.
qudat 5 days ago [-]
I love the design of the website, nice work!
jxyxfinite 5 days ago [-]
Beautiful website! Would love to learn more about the stack
alberth 5 days ago [-]
Monospace font
Please don’t use monospaced fonts for long-form text.
It’s extremely different to read because your eye struggles to see the “shape” of each word, when everything is equally spaced apart.
anthk 5 days ago [-]
As a Unifont user under UXTerm and reading RSS articles, news, chat, code and everything, I think having a good font size matters. WIth Unifont, start from 12pt and use multiples of 4: 16, 20, 24, 28, 32... until the font doesn't look fuzzy and it perfectly suits your resolution/DPI.
Also, as an European I used the read the Teletext (It was almost an electronic newspaper with national/international news/sport results/weather forecasts and so on which worked accesing every page/section by a 3 digit number) on a CRT TV just fine.
Dylan16807 5 days ago [-]
Citation needed.
Rendered at 00:47:42 GMT+0000 (Coordinated Universal Time) with Vercel.
Tons of awesome technical details in the post and I've always felt that Lua's co-routines are incredibly undervalued. We used to run Lua in all sorts of crazy places and it really punches above its weight.
The above quote + zoom out at the end also makes some really good observations. Some of the most powerful systems I've worked on picked a couple core technologies that were complementary with good interop rather than a one-size fits all. Will be interesting to see where Piccolo goes.
The issue is that there exist ways to make a single bytecode instruction take an unbounded amount of time to execute. For example, Lua's "string.find()" is implemented in native code. The interpreter only sees a single OP_CALL opcode, so it will count it as 1 instruction executed. But the actual execution time of the native implementation of string.find() is dependent on its inputs, which are not only variable length strings, but can be maliciously crafted to run in exponential time. Here's an example, shamelessly stolen from Mike Pall himself:
The only way to solve this is to track execution time within the native code as well (i.e. make them fuel-aware), and ensure that they abort immediately if they exhaust the fuel.You will find that this is precisely the approach Piccolo takes.
Piccolo looks amazing, I got a perfect use case for it, and I'm excited for when I get the chance to use it, thank you for working on it.
Is this library offering a lua implementation more well-designed for this use-case? I got all this code to unload the coroutine stack to store and and reload it before continuing it later. Does having C bindings to this library makes sense?
My understanding is that the "stackless" concept here means that it does not store its execution state in the "C runtime stack" (or Rust, in this case).
So, there is some blob of memory describing that info, managed by Piccolo, rather than it residing on the "real" OS execution stack.
In particular, for call chains like: Lua -> C -> Lua -> C (or so), it is normally hard to save/restore the "C" parts of that chain, since you need to peek into the not-normally-accessible and platform-specific C runtime stack details. I wonder: how are you doing it in your system?
In Piccolo, I imagine it would be easier, since even the "-> C ->" portions of the stack are being managed by Piccolo, not the base C runtime. I don't know what the APIs to access that stuff actually look like, though.
Aside: have you looked at Pluto/Eris for Lua serialization of coroutines?
----
EDIT Yes, it seems like the section The "Big Lie" is right up your alley (:
>You should exert care when using this library. Several of its functions violate basic assumptions about Lua code (e.g., that variables local to a function cannot be accessed from outside; that userdata metatables cannot be changed by Lua code; that Lua programs do not crash) and therefore can compromise otherwise secure code (from the manual)
kinda creeps me out.
Please don’t use monospaced fonts for long-form text.
It’s extremely different to read because your eye struggles to see the “shape” of each word, when everything is equally spaced apart.
Also, as an European I used the read the Teletext (It was almost an electronic newspaper with national/international news/sport results/weather forecasts and so on which worked accesing every page/section by a 3 digit number) on a CRT TV just fine.