Does Clojure have a library for "generalized regular expressions" standard regular expression: input = string (pretend its a vector of chars) regex = option ?, repeat *, atleast-one +, way to specify single char / set of chars generalized regular expression: input = vector of arbitrary clojure data regex = option ?, repeat *, atleats-one +, predicates ... that given an element, returns true/fase, indicating whether it's 'in the set' or not is there any library for this?

I guess spec kind of does it, but it'd be nice if there was a library dedicated to this

why do you say spec kind of does that? spec internally uses parsing with derivatives which can parse context free languages which are a super set of regular languages

There might be something like that out there that may be more performance-optimized than spec's current implementation. Certainly I have heard of such things in languages other than Clojure.

I don't know how optimized the implementation is, but Racket has something like this: https://docs.racket-lang.org/reference/match.html

Maybe core.match does this? I haven't used it myself.

@qqq http://clojuredocs.org/clojure.core/subs

The older Java versions, pre 7u6, effectively gave you the choice between reference or copy, by substring always creating a reference, and String constructor allowing you to manually create a copy, if you knew you wanted one. With newer versions of Java, they changed the default, and without a library implementation that works via references you can't get the reference behavior. I suspect they did a lot of performance-testing on real applications before making such an implementation change.

@qqq considered using instaparse ?

Or are you writing a parser to get experience writing parsers?

I asked before, then lost the response. Are the clojure d videos up somewhere?

instaparse only supports strings; I think he was asking about what spec currently does

@tbaldridge: yeah, what @aengelberg said: I want things that can do "regex" over arbitrary data, instead of just strings

"regex" over vector of arbitrary data

one of these days I really should finish my implementation of OMeta in Clojure

is that the VRPI project ?

That sounds like what you're looking for, a generalized parsing system for both strings, and sequences of stuff

?

I believe OMeta was from this lab: http://www.vpri.org/

But yeah, you should finish your OMeta in Clojure project. I feel like 90% of my clojure needs can be solved by "if only @tbaldridge finished those projects ...."

Last I worked on it was before spec existed. I think there's a lot of overlap there. OMeta is a bit like a DSL for spec and conform with polymorphism added

https://en.wikipedia.org/wiki/OMeta is confusing me. Is OMeta a Turing-Complete language, or is it on the same laevel as regex/cfg ?

I recall reading somewhere that due to some design decision, Clojure runs into problems if one has top-level files, i.e. src/a.cljc -> (ns a) results in problems can anyone point me at the article I'm referring to?

It’s not so much Clojure, as it is the JVM

I don’t think that’s a fully valid package name

@madstap thanks 🙂 the logging xf was indeed my best shot before coming back here!

@tbaldridge @aenachronism me too 🙂 and preferably regex builders rather than using the old-school regex language, which is horrible for maintaining code written in it IMO

Hi, I want to get the meta data attached to a function var without knowing the name of the function var at compile time, I tried to do this:

user=> (defn ^::a-meta ttt [])
#'user/ttt
user=> #'ttt
#'user/ttt
user=> (meta ttt)
nil
user=> (meta #'ttt)
{:user/a-meta true, :arglists ([]), :line 11, :column 1, :file "NO_SOURCE_PATH", :name ttt, :ns #object[clojure.lang.Namespace 0x52d645b1 "user"]}
user=> (meta #'(symbol "ttt"))
CompilerException java.lang.ClassCastException: clojure.lang.PersistentList cannot be cast to clojure.lang.Symbol, compiling:(NO_SOURCE_PATH:15:1)

@cmal (meta (var (symbol "ttt")))?

that's not how var works

use resolve

true, ns-resolve is what you want: http://clojuredocs.org/clojure.core/ns-resolve

Thank you. @schmee @bronsa

@fmind thanks

@val_waeselynck coolest stuff https://github.com/vvvvalvalval/scope-capture thanks!

1. most if ... I write are atleast three lines: one for condition, one for then, one for else branch 2. C has test ? true : false 3. does clojure have a nice macro for that ?

if itself?

how is test ? true : false any different than (if test true false)

it's hard to read

how?

when squished on a single line

I like the ? and : to separate the conditions

in practice, do you write single line (if test true-expr false-expr) code ?

yes

maybe I'm just not used to eading them

Is there any standard file reader transducer? Something that can be used to stream-read from a file, that can be used within a transducing process? (meaning e.g. composed with other transducers) (edited) Or should I work up a transducer around lazy file reading myself, i.e. from something like this?

(defn text-file-reader [filepath reader]
  (with-open
    [file-reader (reader filepath)]
    (line-seq file-reader)))

(empty? 23) throws an exception is there a wa to say: (and (.... is a collection) (empty? ...))

there is seq?

also, u can use a seq to check for empty or nil values. (seq []) returns nil

@qqq one of coll? seq? seqable? depending on what you need

Adding to my transduction question three hops above 🙂 is there a core library test function for a newly-written transducer "playing by the rules"? (the rules as per "https://clojure.org/reference/transducers#Creating Transducers")

given something like

(def myv [:a :c :b])
(def mym {:a :anew})

(:c :b :anew)

something like this works but it's a bit insane (keys (clojure.set/rename-keys (zipmap myv (range 3)) mym))

any better alternatives?

time for another Specter solution 😄

(transform ALL #(get mym % %) myv)

ah nice can't use it in this project though

we have medley at least

but not sure it helps in this case

are you not allowed to add dependencies?

user=> (def myv [:a :c :b])
#'user/myv
user=> (def mym {:a :anew})
#'user/mym
user=> (replace mym myv)
[:anew :c :b]

well we have to agree on which to add, and it was agreed not to add specter

I get that people like specter but clojure.core is often more than enough

did not know about replace!

ah nice yes replace is what I needed 😄

there are too many functions in core to remember them all 🙂

btw I also prefer sticking to core as much as possible

I started to learn Specter today, the lib is amazing !

it becomes less scary when you read the List of Macros and List of Navigation on the wiki

and similar thing (still without Specter), from

{:a 1
 :b 2}

{:a inc}

{:a 2
 :b 2}

there is map-values in medley but it's not quite the same

I could split the map many submaps, apply the transformations and merge them all maybe

this works for example

(into {}
      (for [[k v] (seq mm)]
        [k ((k tr identity) v)]))

I think that one’s readable 👍

user=> (def a {:a 1 :b 2})
#'user/a
user=> (def b {:a inc})
#'user/b
user=> (reduce-kv update a b)
{:a 2, :b 2}

nice another point for @bronsa 👍

slick 👍

and last one, from

(def m {:a 1 :b 2})
(def order [:b :a])

I just did

(for [f fields]
    (f m))

mm can just do (map #(% {:a 1 :b 2}) [:b :a]) actually

do you know a function that executes a form and returns nil on Exception ?

I though I knew it, but I can't remember ...

hmmm try

if you really need a fn you can easily wrap it

@mpenet got it

(defmacro ex-nil "Return nil if body throws an Exception"
  [& body] `(try ~@body (catch Exception ex#)))

a bit late to the game but finally got time to take a look at scope-capture - really cool and extremely useful, can definitely see myself using this going forward

is someone familiar with clj-time ? Why is this not valid

(f/parse-local (f/formatter "EEE MMM dd HH:mm:ss yyyy z") "Thu Apr 24 10:27:52 2014 CEST") => Invalid Format: is malformed at "CEST"

I think the last time zone spec needs to be more z's

OK ... apparently, joda time cannot parse 'z' zone name http://www.joda.org/joda-time/key_format.html

wow, did not know that, that's a pretty major omission

yes ... I couldn't agree more !

I think the format was meant for human, not machine

doesn't solve the abbreviated "CEST" parsing, but there is http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormatter.html#withOffsetParsed%28%29

https://stackoverflow.com/questions/1327229/how-can-i-parse-a-date-including-timezone-with-joda-time

@mbjarland thanks for the help, I think there is no answer to deal with the text timezone properly

yeah, seems like it

so this is the "hack" I found:

(def CERT_DATE_PARSER (comp (fn [x] (f/parse (f/formatter "EEE MMM dd HH:mm:ss yyyy") x))
                            (fn [x] (str/replace x #"(\w+ \w+ \d+ [0-9:]+) (\w+) (\d+)" "$1 $3"))))

just wrote something to set the time zone on the formatter, but 1. CEST is not one of the ids and 2. if you get the time as a string it doesn't exactly solve your problem

(def fmt (.withZone (f/formatter "EEE MMM dd HH:mm:ss yyyy") (org.joda.time.DateTimeZone/forID "Europe/Paris")))
(f/parse-local fmt "Thu Apr 24 10:27:52 2014")

@fmind you always have SimpleDateFormat:

(.parse (java.text.SimpleDateFormat. "EEE MMM dd HH:mm:ss yyyy zzz") "Thu Apr 24 10:27:52 2014 CEST")`

it works 🙂 thanks !

I discovered specter yesterday, it's hard not to become addict

I wonder if there is something similar in Python

@mbjarland Try asking in #cursive ? Might get a better response there

Aah, you have 😉

yeah, realized a tad late that there was a channel for cursive

I am trying to implement a algorithm that updates objects in a list. The update function takes each object and the list itself (the other objects) as updated so far. With an imperative approach this is easy using mutability (a for loop and indexes), but what is the best approach to doing this in Clojure? I have tried a reduce, but that of course only passes the original list to the reducing function.

@mac reduce doesn't pass the original list, it passes to function each element and result of previous function evaluation

@mac based on original message I guess map nested in let would work here

@ghsgd2 Yeah, I get that I meant to write the objects (maps in this case) from the original list.

could change elements in function if they're atoms or volatiles

also there's doseq

@mac can you explain a bit more about what the algorithm actually achieves?

The approach in Clojure is usually to take one step back from the (traditional) implementation and think about the problem holistically instead.

@seancorfield I can. It updates the position of planets. The new position of a planet depends on its own previous position and the position of the other planets - new position if they have already been updated and old if they have not.

@seancorfield This is the js equivalent: https://gist.github.com/maacl/b958c97a417527f4ee4dad19b361ae9f

(reduce-kv (fn [planets i planet] (assoc planets i (new-pos planet planets))) planets planets)

you can use planets as both accumulator and collection over which to reduce

(defn new-position [solar-system planet]
  ...)
(defn update-solar-system [solar-system-state]
  (map (partial new-position state) solar-system-state))

(def universe-time (iterate update-solar-system initial-universe))

depending on how you store the state of the solar system. as a list, a map, etc

(defn reposition [planets]
    (map (partial new-position planets) planets))

but you have to define new-position to take the list of planets first

no, using map is not correct

note that in @mac's js version, each invocation of newPosition takes the updated planets map, not the original one

good point.

reduce is the natural way to express this in clojure

@bronsa @dpsutton agreed, I though the planet object did not change after each iteration

@bronsa Yes, that is correct. This is actually what tripped me up. Planets is a vector of maps, but reduce-kv works anyway as far as I can tell.

yeah reduce-kv supports vectors

reduce is the traditional solution for state + rules for state change => new state @mac

(for a given set of events -- in this case the set of planets acts as both the initial state and the set of "events" that trigger state change)

@seancorfield Yes, I was missing the idea of using planets as both the accumulator and collection over which to reduce, as mentioned by @bronsa.

@mac are you doing gravitational dynamics?

because if so you're making it unnecessarily difficult from a programming standpoint

since from a physics standpoint you'd like to do one pass over the planets and calculate force

and then another pass to update momentum

and another pass to update position

although for speed probably the best thing to do is to solve for the gravitational potential on a grid and then interpolate to get forces, depending on how many planets you have

i doubt there's a good poisson library in javascript, haha

@adamvh I am, I am trying to reimplement this: https://github.com/raisoman/de-for-children/blob/master/processingjs/p5/mars-trip.html in cljs.

ya, from a physics POV, that code would be better if it accumulated xpp and ypp in arrays

and only actually moved the planets after it had calculated that for all of them

splitting hairs, probably; the errors from doing it as linked take many orbits to be visible

(comp (map #(update-position planets)) (map get-force))

would give you pretty much what you need

@adamvh I also had a sneaking suspicion that my original was actually more correct, but the constants are calibrated to work with the js version, which is quite sensitive to starting conditions since it is inserting a tiny space-ship into orbit around mars. Not that I really understand the physics very well 🙂

(def queue (agent []
                  :error-mode :continue
                  :error-handler (fn [a ex]
                                   (l/error ex "slack agent failed"))))

(send-off ... (retry fn) ...)

(defn retry [f] (fn x [& args] (try (apply f args) (catch Throwable t (apply x args)))))

inside a (fn [] .... ) is there a way to "refer to self, i.e. the function being defined" -- and this is not via recur

Why are you not able to use recur?

you can name a fn (fn f [] …)

because I need to call the function two/three times, in a tree like fashion, it's not a loop

yeah, that’s why the (fn f [] ...) syntax exists, it’s the straightforward way to do this

though if two functions need to do this mutually, letfn exists for that purpose and no other

ah, the (fn f [] ...) syntax gives it a name I can refer to it by; no idea howI missed it all these years; thanks!

extra bonus - when you use (fn f [] …) the name “f” will show up in a stack trace

sometimes I name my fn even if it doesn’t self call just so my stack traces will make a little more sense

it makes me want versions of partial and comp and complement and fnil etc. that take a “name” parameter too frankly

How does the keyword-as-accessor syntax actually work in the language? E.g. (:b {:a 1, :b 2}) Is this just special syntax built into the language or is there some reason why keywords can act as functions in this way?

it is not a syntax, keywords are functions

symbols, hash-maps, sets, and vectors are also functions

at one point they argued about making regexes functions too but they opted not to do that

specifically, keywords implement IFn

@chris right when I say “is a function” about clojure that means “implements IFn”

But there must be something special about them because there are an infinite number of them and they spring forth whenever you use them

@Justin how’s that different from fn?

is that different from how a function works?

haha

or a symbol

the reader interns symbols/keywords

but symbols can have metadata so they do something different about the interning right?

yeah symbols just have their internal string interned

actually since a few versions ago that might not be true, I can't remember

and interning has nothing to do with being used as a function

true, I was just answering how they "pop out of existence"

(foo a b c) in clojure is something like ((IFn)foo).invoke(a, b, c) in java

I previously just thought of them as identifiers that had some concrete unique representation that allowed them to be used as keys in a map. But then they can also be used in the first position in a list (which I think of as “function” position but I’m sure it has a more precise name). I suppose the point is that they are instantiated as lookup functions the moment the reader sees them and the reference to that lookup function is what is used as a key?

they are instances of clojure.lang.Keyword

which implements IFn

clojure.lang.IFn

oh right. okay got it

thanks all

@Justin also, regarding “allowed to use them as keys in a map” that’s true of literally any value in clojure

as an aside

(:foo {:foo 1}) becomes something like new Keyword ("foo").invoke ( {:foo 1 })

any object on the jvm that doesn't replace Object's hashCode and equals with methods that throw execeptions

which is implemented roughly as {:foo 1}.get(this)

clojure values just make better keys then most objects on the jvm

immutability is a good idea™

ahh right- you can use a mutable thing as a key, but if the mutation changes the hash or equality you will have a bad time haha

@noisesmith Right that was phrased awkwardly. Keywords are a little interesting because unlike something like a let binding or a formal parameter, what you call the keyword actually changes something in memory. I guess that’s no different from a string literal but it has always felt unusual to me because a keyword looks like a formal name rather than data.

At least I think that’s right.

keywords are data, not labels

just like a string or a number

same with symbols

Thanks yea that’s important to remember. Because to my C/C++/JS mind they look like names.

(in fact they are listed under data_structures on http://clojure.org, https://clojure.org/reference/data_structures)

apart from reader macros, there's no labels or language syntax that's not reified as a value

let bindings are sneaky to find, and even sneakier to do anything with as data haha but yeah https://gist.github.com/noisesmith/3490f2d3ed98e294e033b002bc2de178

omg

Beyond the (keys &env) I wouldn’t really consider let-bindings to be “reified” (officially) in clj

right - but that snippet proves they are reified

sort of

hah

no

you are reading compiler state using the macro and generating code to reproduce it at runtime

oh… that is different OK

what bronsa meant is (let [a 1] a) is not "syntax" as such, it is a list containing a symbol, a vector, and another symbol

user=> ((fn [x] ((x {:a {:a 1}}) x)) :a)
1
user=> 

anyone know why

:1

:foo/1

( or maybe they’re both invalid and :1 just happens to work? )

correct

they are both invalid literals

:foo/1 happens to work but it's still invalid

https://clojure.org/reference/reader - yeah my “read” of this is that :1 is illegal but accidentally works in some cases

well, no, right? because the reader doesn't determine what a valid keyword is

yep, I said literal

maybe a strict reading of that reference would also imply : is bad, and keywords of that form are used in core libs (and it’s neither here nor there but the reader does accept them)

it was unsettling to see that being used on purprose

If an implementation doesn't stop you, someone somewhere will do it. 🙂

and think it is a great idea

there is a lot of background on this - the regex for keywords in the reader is actually wrong, which is why it works

we “fixed” it in 1.6 and found out it broke a lot of existing code

so we unfixed it

Taking advantage of such behavior does put you at some kind of risk of a future version of Clojure reserving characters for other purposes, and now either you update your application or don't run it or its data with that version of Clojure. Given sensitivity to backwards compatibility by core team, I doubt they would break such things for fun.

I’d say at this point that there is no chance we will take that away

we did break it, and it wasn’t fun

I notice one can now override *reader-resolver* in Clojure 1.9. Has anyone tried this? What scenarios is it useful for?

If I wanted to write a utility that overrode *reader-resolver* to do cool stuff, I would have to alter-var-root it rather than bindings it so that I can load code with that new reader resolver in place, right?

it’s useful if you want to have control over how namespace-sensitive things resolve (like autoresolved keywords)

Looks like I can affect the behavior of read-string:

user=> (binding [*reader-resolver* (reify clojure.lang.LispReader$Resolver (resolveAlias [this sym] 'blah))] (read-string "::a/b"))
:blah/b

It seems to me that that's the only thing I can do with binding

Just want to make sure that's accurate

yes

really best for programmatic reader cases where you are creating the environment (not for general code loading in the RT)

gotcha, that's what I wanted to confirm

thanks

I was hoping I could write myself a with-aliases macro that lets me define arbitrary aliases in local scope or something

I think reading has already been done at macro time so that wouldn’t work

yep

yep

unless I use push-thread-bindings and pop-thread-bindings....

then I could write a defalias macro that pushes a new *reader-resolver* for the remainder of that file

but then I don't know when to pop it

having a push-thread-binding escape its lexical context is not a good idea anyway

(ns my.ns)

(defalias o other.ns)

::o/a-keyword

(end-aliases!)

if your push-thread-binding gets executed from within a binding, that binding ill pop the wrong thing

if you are going to be gross, hijack the dispatchMacros table in the reader

haha

brb forking Clojure

Am I missing something, or does Clojure not have a way to refer to a piece of code from a docstring? I’m hoping to hyperlink between namespaces when an implicit dependency exists. I’ve considered using #_ macros but that invokes a (requires) clause I’d rather avoid.

IDE stack is Cursive with a close fallback of Emacs/etc

you could use metadata

@briantrice there is no standardized docstring markup for referring to symbols.

instead of encoding it in the doc string and then having a way to parse it out

you could just be explicit with meta data (defn ^{:refers-to 'other.name-space} foo [] 42)

@arrdem okay! glad to know it for sure

Some authors use Markdown style backtick blocks. I personally like to use #'foo or #'other-ns/foo var notation in backtick blocks in docstrings.

@smith.adriane okay that is interesting. I can do it without require, then? I’ll see whether cursive/etc pick up on it

Some of the tools support wiki-style [[my.other/thing]] references.

#' is also a good point, thanks

lessee…

the meta-data keyword is completely made up by me, so if you wanted ide support, you’d have to add it

ah

just trying to say that this isn’t a standardized practice. just a suggestion for how you could start one if you wanted

Cursive is trying hard to resolve it, though. just failing…

hm. of course the real problem is that this is across Clojure/ClojureScript code where I can’t exactly write cljc.

maybe I will find a way to cljc it, but for now, I have to keep the people reading the code aware.

Where would I RFC on such a thing?

I’ll take it to #cursive in case it appeals…

Hey all, I'm wondering in clojure since there aren't really object methods how do I know what functions I can use on something?

For example, how do I know all the methods I can use on a channel made from core.async's (chan)?

Clojure seems to follow this philosophy: https://stackoverflow.com/questions/6016271/why-is-it-better-to-have-100-functions-operate-on-one-data-structure-than-10-fun

thanks, but that doesn't really answer my question. For example, I'd expect from that philosophy to be able to do "nth", "count", "map", etc

But none of these actually work so I'm wondering what are the methods that work for chan

I'm not sure if this helps either: you can lookup all the protocols that chan implements, then look at functions that depend on those protocols

also, I'm pretty sure you can do something map/transducer like on chans

for channels specifically, most of the functions that work with channels can be found at https://clojure.github.io/core.async/

hey guys, is there any file reader transducer in any core library? Something that can be used to stream-read from a file, as part of a transducing process? Or should I work up a transducer around lazy file reading myself (which is what I'm about to be doing now)

@matan not sure is worth a lib, is this something useful? https://gist.github.com/reborg/2d9b31f2209f9b3afd1dc401b3dffc2c#file-split-by-clj-L55

@qqq I'm not really sure what you mean. How would Iook up all the protocols for channel?

@smith.adriane Those are just functions within the core.async namespace, NOT functions that I can use on a ManyToManyChannel

@reborg yes, but doesn't that snippet show a transduction that is specific to the file reader? doesn't it go against the motivation for transducers? as in being generically applicable regardless of the actual input source

@derpocious, there’s not really a great way to obtain a list of every function that takes a ManyToMany channel as it’s first argument

@reborg mmm on second thought maybe you are right and this is just trivial

whynot

a couple things, 1) ManyToManyChannel is a concrete type, https://github.com/clojure/core.async/blob/0498ba69fda5d930e3e7568cc90ca933c19c42ca/src/main/clojure/cljs/core/async/impl/channels.cljs#L30

and you often want to program against abstract types instead of concrete types

2) since clojure’s protocol system is open

at runtime, more protocols can be extended to work with the concrete type of ManyToManyChannel

3) in addition, anyone could write a function that takes a ManyToManyChannel as it’s first argument

or if the ManyToManyChannel was added to more protocols, then at any point, any protocol can start working with ManyToManyChannel

for most clojure functions, the types that functions expect aren’t annotated anywhere

so even if you went through every function in every namespace, there’s not typically a good way to ask if a particular function accepts a particular type

basically, clojure tends to adopt an approach that is dynamic and open which makes it more difficult to answer questions like “which functions work with the type?”

but makes a lot of other designs easier

I was under the impression that things like (take 1 (drop 40000000 (iterate inc 1))) would take constant memory but it doesn’t appear so. Could somebody explain what exactly happens here?

I get “Out of memory” on a constrained system while running this

@nooga, how constrained?

it’s http://repl.it so I have no idea, but the memory assigned there must be somewhat small

but that shoudn’t matter if my assumption that this will take constant memory is correct

I think it does take constant memory

my thinking was that iterate will return a lazy seq, drop will basically throw away elements as it goes and first will just cut the remaining lazy-seq and everything will be garbage collected

and those 40M elements won’t be allocated anywhere

that was my impression as well

the one common issue is “hanging onto the head”

I fail to see that in my example

https://repl.it/repls/CultivatedAridStegosaurus

for eg. (def xs (take 1 (drop 40000000 (iterate inc 1)))

sometimes the repl messes things up

since it assigns the whole sequence to *1

which means it is hanging onto the head

try (first (take 1 (drop 40000000 (iterate inc 1))))

but (println (take 1 (drop 40000000 (iterate inc 1)))) will also run out of memory

same

hm

puzzling

will it?

it does

on the http://repl.it site

oh

who knows what http://repl.it does

as someone who runs a publicly accessible code sandbox, I can tell you they are never vanilla and extrapolating behavior to the "real" thing is fraught