Fun to see someone make a complete while-loop macro. Occasionally I get tempted to make one, but almost always find it's better to rethink a while loop into a Stream or just a for loop if possible to keep my logic in-place.
The Python version is pretty simple, but really isn't any shorter. It also intertwines the timing logic and node ping actions. Although it takes a bit of practice to become fluent with the stream approach, it's much more compose-able. Take the example in the article where you end up wanting to run a bunch of functions with a timeout on some interval, you can cut the middle 2 lines into a new function in a utility library. Then you could rewrite the code like (notice it's just cutting and pasting complete lines):
Usually re-writing something into this form makes me think of many programming tasks as a set of discrete steps. Do the first stream, then add the second step, and so forth. This tends to make it easier to dive back into code after a while as each step has to be incremental. Also a side note, if you _really_ want to cheat and need to use a mutable variable, you can use (abuse?) `Process.get/1` and `Process.put/2` with a for-loop. CouchDB uses a similar idea to create stateful iterators if I recall the code correctly.
bitwalker 1611 days ago [-]
Good points! The time where the `while` loop is actually better is very situational, but when it is a better fit, it is definitely handy to have in your back pocket. But more generally I just think it is useful to know how you can implement language constructs like this in Elixir with nothing more than the primitives it provides, some macro glue, and a bit of creativity :)
A couple things though; one issue with your first example is that you incur a 1s wait for either success or failure conditions, where what you typically want is for only the failure condition to impose that cost. Luckily, you can keep this approach and fix it by removing `Stream.zip(Stream.interval(1_000))` and replacing it with `Stream.intersperse(Stream.interval(1_000))`.
The other problem though is that you end up incurring quite a few additional function calls vs the `while`, since the `while` inlines the predicate and associated machinery. You end up with precisely one function call per iteration, with the ability to exit early on failure or timeout, whereas the function-based form is going to incur potentially several function calls, depending on how the `Stream` is being constructed - if I recall implementation details correctly, your first example would be a minimum of 3 calls per iteration; one for the unfold, one for the interval, and one for the take_while.
Of course, that's all really just optimization, and your `Stream` based solution is perfectly fine in general. The only additional benefit (IMO) to the `while` form, is that it reads very succinctly, and due to most being familiar with imperative constructs like that, they can read it with virtually no effort, where reading the `Stream` form takes some care to understand what you get at each step, and what it produces in the end.
RE: Mutability, I think I mentioned it in the post, but I really didn't want _actual_ mutability, just the appearance of it. So while one could use the process dictionary, or ETS, or even another process, to get something like "real" mutability, that kind of defeats the point ;)
elcritch 1610 days ago [-]
It was a good read! Especially since you went through all the details. Handling different inputs and alternates like `after` are a bit tricky. And you're right on the downsides of the "stream" style. There's a bit of overhead compared to the macro form which can inline things a bit. It also takes a couple of more "cognitive" cycles to write, but usually I can read the results more quickly later.
In general I do wish Elixir provided a `reduce` macro similar to the for-loop one with the ability to halt. While-loop logic is easier to refashion as a reduce but it's harder to recall the argument orders for various Enum.reduce forms.
Do you use this while-macro in production code or just test code? Yah the "real" mutability tricks are more for those wondering if you _can_ do it. Sometimes it's not obvious, but one of the nice things in BEAM is that you can cheat.
P.S. thanks for all of your Elixir tools!
bitwalker 1609 days ago [-]
> Do you use this while-macro in production code or just test code?
I use the `while` macro in the Distillery tests, since it made expressing certain tests much less verbose - namely those dealing with building up a release, spinning it up in a new node, and then verifying conditions on the running node meet expectations. In general though, I don't. I haven't needed it elsewhere (well, I have one other library I have considered using it in, but haven't yet). I would be hesitant to introduce it without a clear need, just because it would likely catch other developers off guard unless clearly documented.
To be clear though, I would support something like it in the language, but also understand why it isn't there; there just isn't a clear enough need, and as you've demonstrated, `Stream` can more or less be used to cover the general cases.
I think this was a fantastic blog post that really showed off the power of macros. It's the one area of Elixir I haven't tried to delve into. Being able to see the author's thought process really helped me understand what was happening too. Kudos!
bitwalker 1611 days ago [-]
Thanks! I don't write very often, since I don't really know if people get anything out of it, but feedback like this is definitely encouraging :)
HorizonXP 1611 days ago [-]
I need to start writing too. I just subscribed to your RSS feed.
davidw 1611 days ago [-]
This is definitely something different from Erlang, where clarity and ease of figuring things out win out over fancy coding.
let limit = secondsToDiffTime 60
start <- getCurrentTime
iterateUntil id $ andM
[ someApplicationLogicReturningBoolean
, do
now <- getCurrentTime
return $ (diffUtcTime start now) >= limit
]
This is a really rough sketch from someone who just looked all this up, of course :-)
Recursion is probably still much more natural here.
galaxyLogic 1611 days ago [-]
Seems quite complicated. Why not just use while-loop? :-)
(in other words a language which provides it)
sixstring982 1611 days ago [-]
Or just use recursion?
But hey, this article is an interesting way to get around that!
bitwalker 1611 days ago [-]
Well, I pointed out at the beginning that one can use recursion, and the implementation of the `while` construct itself uses recursion as well (in fact, you can't implement `while` without it in Elixir).
In any case, expressing the equivalent of a `while` loop with predicates and timeouts is quite syntactically noisy in Elixir - it is certainly doable, but much less clear than the imperative equivalent.
vendiddy 1611 days ago [-]
According to him, it was an experiment:
> I’m not sure I would recommend using this macro generally, but it was a fun experiment, and I love the opportunity to stretch the macro muscle a bit, especially with one like this which really shows how you extend the syntax of the language via macros
In practice, I'd probably use `Enum.reduce` or recursion for these kinds of things Elixir.
galaxyLogic 1610 days ago [-]
I got that. The more basic question I am interested in is why doesn't Elixir have while-loop? Is that a missing feature in all "functional languages"? And if so, why is that?
bitwalker 1609 days ago [-]
It just isn't needed, most things can be expressed using nothing more than recursion; more complex things can be expressed using `for` comprehensions, or by composing the various `Stream` APIs.
I think `while` can more succinctly express certain patterns, but that window is pretty narrow, so adding an additional keyword to the language that has so much overlap with other options just doesn't make sense.
Elixir is a little unique in that it has no mutability, so fundamentally a `while` loop isn't really a thing that makes sense - you can't mutate anything in the containing scope, so you are left with basically nothing useful you can do. The macro I build in this post is a clever way of making it work, but only with the bindings given to the macro, and it isn't actually mutating them.
For other FP languages that are immutable, they likely don't have a `while`, or provide some kind of machinery that emulates it (like Haskell's monadic loops). For FP languages with opt-in mutability, they might have `while`, just depends on the language.
vendiddy 1607 days ago [-]
Functional languages rest on two main principles: (1) Immutability (2) Transforming inputs to outputs
Shared mutable states creates a lot of accidental complexity and makes writing concurrent applications harder. I recommend the talk "The Value of Values" by Rich Hickey for a more in depth analysis.
A traditional while loop like while (x < 5) { ... } involves mutating state, so it's not possible to implement in a purely functional language. In the case of bitwalker's implementation, a value is returned to work around this limitation.
cpursley 1611 days ago [-]
Languages that do provide a while loop are generally OOP, which have their own class (see what I did there) of foot-guns.
np_tedious 1611 days ago [-]
No I don't get it. Explain plz
dnh44 1611 days ago [-]
The comment only makes sense if you are aware of the English language idiom "shooting yourself in the foot" which means to accidentally hurt yourself.
So the comment is saying that while-loops are generally a feature of object oriented languages, which have many tools (they use the term "foot-gun as a joke) that make it easy to accidentally make mistakes, so writing your own while-loop in Elixir isn't as crazy as it might seem.
np_tedious 1609 days ago [-]
Sorry, my comment was dumb lazy sarcasm. The joke was more around class/OOP as I read it.
bitwalker 1611 days ago [-]
Sure, the implementation of the macro is complicated, but the actual `while` construct that results is just as expressive as that of any language with a native `while`.
But that's all beside the point, this was just an exercise, and was useful in a project where the reduction in complexity it provided was handy
Rendered at 06:41:50 GMT+0000 (Coordinated Universal Time) with Vercel.
Take:
Could be written: Which in Python would probably be something like (not using the Walrus, and made up system lib names): The Python version is pretty simple, but really isn't any shorter. It also intertwines the timing logic and node ping actions. Although it takes a bit of practice to become fluent with the stream approach, it's much more compose-able. Take the example in the article where you end up wanting to run a bunch of functions with a timeout on some interval, you can cut the middle 2 lines into a new function in a utility library. Then you could rewrite the code like (notice it's just cutting and pasting complete lines): Usually re-writing something into this form makes me think of many programming tasks as a set of discrete steps. Do the first stream, then add the second step, and so forth. This tends to make it easier to dive back into code after a while as each step has to be incremental. Also a side note, if you _really_ want to cheat and need to use a mutable variable, you can use (abuse?) `Process.get/1` and `Process.put/2` with a for-loop. CouchDB uses a similar idea to create stateful iterators if I recall the code correctly.A couple things though; one issue with your first example is that you incur a 1s wait for either success or failure conditions, where what you typically want is for only the failure condition to impose that cost. Luckily, you can keep this approach and fix it by removing `Stream.zip(Stream.interval(1_000))` and replacing it with `Stream.intersperse(Stream.interval(1_000))`.
The other problem though is that you end up incurring quite a few additional function calls vs the `while`, since the `while` inlines the predicate and associated machinery. You end up with precisely one function call per iteration, with the ability to exit early on failure or timeout, whereas the function-based form is going to incur potentially several function calls, depending on how the `Stream` is being constructed - if I recall implementation details correctly, your first example would be a minimum of 3 calls per iteration; one for the unfold, one for the interval, and one for the take_while.
Of course, that's all really just optimization, and your `Stream` based solution is perfectly fine in general. The only additional benefit (IMO) to the `while` form, is that it reads very succinctly, and due to most being familiar with imperative constructs like that, they can read it with virtually no effort, where reading the `Stream` form takes some care to understand what you get at each step, and what it produces in the end.
RE: Mutability, I think I mentioned it in the post, but I really didn't want _actual_ mutability, just the appearance of it. So while one could use the process dictionary, or ETS, or even another process, to get something like "real" mutability, that kind of defeats the point ;)
In general I do wish Elixir provided a `reduce` macro similar to the for-loop one with the ability to halt. While-loop logic is easier to refashion as a reduce but it's harder to recall the argument orders for various Enum.reduce forms.
Do you use this while-macro in production code or just test code? Yah the "real" mutability tricks are more for those wondering if you _can_ do it. Sometimes it's not obvious, but one of the nice things in BEAM is that you can cheat.
P.S. thanks for all of your Elixir tools!
I use the `while` macro in the Distillery tests, since it made expressing certain tests much less verbose - namely those dealing with building up a release, spinning it up in a new node, and then verifying conditions on the running node meet expectations. In general though, I don't. I haven't needed it elsewhere (well, I have one other library I have considered using it in, but haven't yet). I would be hesitant to introduce it without a clear need, just because it would likely catch other developers off guard unless clearly documented.
To be clear though, I would support something like it in the language, but also understand why it isn't there; there just isn't a clear enough need, and as you've demonstrated, `Stream` can more or less be used to cover the general cases.
> P.S. thanks for all of your Elixir tools!
Thanks for the kind words!
Recursion is probably still much more natural here.
(in other words a language which provides it)
But hey, this article is an interesting way to get around that!
In any case, expressing the equivalent of a `while` loop with predicates and timeouts is quite syntactically noisy in Elixir - it is certainly doable, but much less clear than the imperative equivalent.
> I’m not sure I would recommend using this macro generally, but it was a fun experiment, and I love the opportunity to stretch the macro muscle a bit, especially with one like this which really shows how you extend the syntax of the language via macros
In practice, I'd probably use `Enum.reduce` or recursion for these kinds of things Elixir.
I think `while` can more succinctly express certain patterns, but that window is pretty narrow, so adding an additional keyword to the language that has so much overlap with other options just doesn't make sense.
Elixir is a little unique in that it has no mutability, so fundamentally a `while` loop isn't really a thing that makes sense - you can't mutate anything in the containing scope, so you are left with basically nothing useful you can do. The macro I build in this post is a clever way of making it work, but only with the bindings given to the macro, and it isn't actually mutating them.
For other FP languages that are immutable, they likely don't have a `while`, or provide some kind of machinery that emulates it (like Haskell's monadic loops). For FP languages with opt-in mutability, they might have `while`, just depends on the language.
Shared mutable states creates a lot of accidental complexity and makes writing concurrent applications harder. I recommend the talk "The Value of Values" by Rich Hickey for a more in depth analysis.
A traditional while loop like while (x < 5) { ... } involves mutating state, so it's not possible to implement in a purely functional language. In the case of bitwalker's implementation, a value is returned to work around this limitation.
So the comment is saying that while-loops are generally a feature of object oriented languages, which have many tools (they use the term "foot-gun as a joke) that make it easy to accidentally make mistakes, so writing your own while-loop in Elixir isn't as crazy as it might seem.
But that's all beside the point, this was just an exercise, and was useful in a project where the reduction in complexity it provided was handy