Fork me on GitHub
Drew Verlee00:10:59

i would have expected ["foo" ""] from (str/split "foo/" #"/") maybe im used to how python does this.


fwiw, this does what you want:

(clojure.string/split "foo/" #"/" -1)
["foo" ""]

Drew Verlee00:10:51

I find it odd i can't figure out a way to verify is a string is a valid path where i could write a to.

Drew Verlee00:10:35

Like, it seems your best bet is to try to write the file and catch the exception.


the java File object has .exists and .isDirectory that you can use

Drew Verlee00:10:50

Yea. But i want to know if the program can write to a location. its getting kind of tricky. "/" => nope "A" => yep "valid/path/foo" => yep "invalid/path/foo" => nope "valid/path/" => nope (no file name)


@drewverlee maybe something like this?

+user=> (defn valid-file? [f] (let [file ( f)] (or (.isDirectory file) (.exists (.getParentFile file)))))
+user=> (valid-file? "/etc/passwd")
+user=> (valid-file? "/etc/passwd-not-on-disk")
+user=> (valid-file? "/etc-no-such-dir/passwd-not-on-disk")


oh, now that you clarify, make it (and (not (.isDirectory file)) (.exists (.getParentFile file)))


oh - that still doesn't work for "A"


although, there might be other reasons you can’t write to the file


like out of space or permissions

Drew Verlee00:10:16

Yea. I agree, checking everything would be somewhat involved and mean a lot of trade offs. I was just curious about the basic stuff concerning file paths

Drew Verlee00:10:44

This is what i have so far:

(s/def ::valid-file?
   #(not (str/blank? %))
    :can-write               #(.canWrite (io/file %))
    :in-current-dir          #(nil? (.getParentFile (io/file %)))
    :valid-file-path         (s/and
                              #(.isDirectory (.getParentFile (io/file %)))
                              #(not (str/blank? (last (str/split % #"/"))))))))

Drew Verlee00:10:11

the last condition doesn't wok because of how string split works.


it looks like this one meets your spec

+user=> (defn valid-file? [f] (let [file ( f)] (and (not (.isDirectory file)) (.exists (.getParentFile (.getCanonicalFile file))))))
+user=> (valid-file? "A")
+user=> (valid-file? "A/B")
+user=> (valid-file? "project.clj")

Drew Verlee01:10:34

Thanks! Ill admit i felt like getCanonicalFile might have been the missing piece to making this easy, but i couldn't figure out what it did. Thane name and docs seem to assume some background i dont have.


the canonicalFile is absolute and has a parent (even if it's implicit in the original) while the raw file is dependent on your pwd


i don’t know a lot about your use case, but it seems like that spec definition might cause headaches later


my intuition is that if some data conforms to a spec, it should always conform to the spec


oh - yeah, that's highly contingent isn't it


and not depend on things like what’s on the filesystem where the spec code is being run

Drew Verlee00:10:05

Why would that limitation be helpful.

Drew Verlee00:10:43

It will make that spec somewhat meaningless for generators, i admit.


it makes the spec function not pure anymore


which means that if you, for example, use spec to check conformance on a build server and it fails


then you don’t know if it’s a problem with the data you’re checking or if there’s some setup problem on your build server


in other words, it means that any code that uses that spec now has the implicit argument of the filesystem


and it’s generally harder to test implicit arguments

Drew Verlee00:10:12

Sure. But the thing im trying to verify depends on the state of the filesystem.

Drew Verlee00:10:11

My plan was to verify inputs with it, so that i could give better feedback to the user.


yea, it does seem like a useful thing to specify, that your function expects a writable file path


or, as another example, your function expects an active socket connection or something


ok, nevermind. seems like a useful thing to do


i wonder if there’s a convention for saying your spec has implicit arguments

Drew Verlee00:10:20

I'm glad you brought that up, its worth thinking about.


like ::valid-file?!


is protobuf a good choice for clojrue or not? is anyone using it?


@rcustodio What problem are you trying to solve that you think protobuf might be (part of) the solution for?


Communication over rmq / tcp


Between arbitrary processes? Or between Clojure processes?


I've found transit very useful - it's faster than edn, and supports all the clojure data types, and the customization for extra types is first class and local instead of implicit and global


I guess it will be between clojure processes only, idk if the project will adopt others Lang in the future


Just the api and websocket is json, but that is easy to converti


If you're doing Clojure-to-Clojure for now, I'd stick to EDN for the first version, then switch to Transit and see if it improves performance etc.


For EDN, there are other language readers/writers. I assume there are for Transit too?


What is this transit? Is it possible to pass it through rmq?


It's a pass-through-able as Protobuf 🙂


I know few about edn as well


I just know the edn file


Hmm, i will research about it


"Transit is supported in Ruby, Python, Clojure, JavaScript, Java."


Thanks ✌️:skin-tone-2:✌️:skin-tone-2:✌️:skin-tone-2:


yeah, transit might have better multi language support than edn, and it's definitely better performing and more flexible compared to using pr-str and custom reader tags or whatever


And is better and more able to use in clojure than protobuf?


a lot easier to use, at least


I see, thanks

Michael Stokley04:10:03

is it good practice to set up infinite "go" processes to listen to channels?

Michael Stokley04:10:13

e. g. (go (while true (some-func (<! input-channel)))

Michael Stokley04:10:35

i tried (close! channel) but it doesn't seem to help.


a closed channel won't interrupt a loop that reads from it - if you read from a closed channel it instantly returns nil, and the consumer should be checking for that

Michael Stokley04:10:42

cool, i can do that


(go (when-some [v (<! input-channel)] (some-func v) (recur)))

Michael Stokley04:10:49

i'm traversing a graph asynchronously. i need a way of keeping track of visited nodes. would an atom be a suitable ref type for that?


that seems reasonable, yeah - just be aware that if you hit conflicts (two or more threads trying to alter the value at once), you will end up seeing retries. You'll want to make sure that the inevitable retries don't break things with your core.async usage, and if performance matters at all you probably want to do batching of some sort, or limit the recursion branches somehow

Michael Stokley04:10:18

i think it should be because i don't need - i don't need any kind of transaction logic


with something like graph traversal, the retries can easily be more expensive than the navigation is, making the parallel version slower than the single threaded synchronous one


it's faster to look up a value in a hash and put it in a set, than it is to look up a value in a hash, create a mutation checkpoint, attempt to put the value in a hash, check the checkpoint, and either retry or return the new value


but I don't know what your data looks like or how it is acquired, only testing will tell you what performs best

Michael Stokley04:10:30

it's urls. it's a scraping project.


@michael740 specifically the big problem is with parallelizing something like that is that the lookups of nodes in the graph is so much faster than the mutating transactions that the larger the number of threads, the larger of your percentage of time is spent on automatic retries, so the slower the whole thing gets


OK - with URLs that's different - try different parallelisms to see what has the best throughput though

Michael Stokley04:10:57

so i send a request, parse the html for more links to fetch, go fetch them in parallel, ...


also you could check out memoize - if you memoize the "get links from URL" function that might simplify some of your logic and eliminate retries


but then again there's no predicate to see if a memoize has seen an arg already, so that's probably a bad choice

Michael Stokley04:10:54

if i have a get-links-from-url function and i've seen the url already, i think i just want to move on.


right, that's fair - and I checked and memoize is just doing a swap! anyway

Michael Stokley04:10:21

i appreciate your help!

Hendrik Poernama08:10:52

is there an 'identity' transducer? I'm thinking (map identity) if nothing else.


the identity transducer is identity


hi, im new to programming, but i want to start from clj, plz give me some advice where to start ?


clojure for the brave and true


i heard about it, is this the most understandable way to start?


I think it's likely to be the most approachable way to begin programming


SICP is a great book, but likely to be less approachable unless you're into math


I didn't realize that was the SICP class, I thought it was the book


yeah, its the videos, and has course assignments 🙂


thnx, other thoughts are welcome )


if you go with sicp


please take reaaaaaally small bites, especially if you're new to programming


make sure you get what you're reading


i found & really interesting, thnx!


there are more in the series, by the way


cool, 10x


@seancorfield and @noisesmith I’m checking out about transit to communication, now to validate data is schema a good choice? (


I would look at clojure.spec instead


Yes, definitely look at clojure.spec first as that is "built-in" to Clojure 1.9.


Since I’m still developing my app, it will be out march or later, is it wise to use clojure 1.9?


It's essentially in release candidate status now


So probably it will be final until then?


Alex said they plan no changes between Beta 4 and "gold", except to fix any regression bugs introduced.


Great, so I will start to use it in my new projects


We're using Beta 3 in production, with Beta 4 in testing right now.


(we've had 1.9 in production since one of the first Alpha builds)


Woow, okey, thanks that helps a lot


if you're going to do sicp, there's a really great online version

Joe Lane17:10:00

Hey friends, I’m aware of destructuring with keys which allows me to write

(defn foo [{:keys [foo bar baz]} some-map]
    (println foo bar baz))
I was wondering if the inverse of destructuring, (restructuring?) exists or if I’m just googleing the wrong name or if this is even a good idea at all. I’m thinking it would look like this
user> (let [foo "foo"         ;; This let is just used to create lexically scoped values
        bar "my-bar"
        baz [1 2 3]]
(->map {:keys [foo bar baz]})) ;; or some other syntax, I didn't put a ton of thought into it.
=> {:foo "foo" :bar "my-bar" :baz [1 2 3]}
Am I totally missing something?


I have a gist for that

Joe Lane17:10:50

thanks @noisesmith, it looks like it captures all locals, which isn’t quite what I was looking for but this looks super cool.


it’s easy to combine it with select-keys


(or just write a macro that expands to a map with a key for each symbol)

Joe Lane17:10:29

great, I see what you’re saying now. Thanks @noisesmith


(ins)user=> (defmacro restructure [syms] (into {} (for [sym syms] [(keyword (name sym)) sym])))
(ins)user=> (def a 0)
(ins)user=> (def b 1)
(ins)user=> (let [c 2] (restructure [a b c]))
{:a 0, :b 1, :c 2}


it’s only so often you can write a macro with no quoting, heh

Joe Lane17:10:26

Wow. It’s pretty cool to see you work through this stuff.


what's idiomatic way to name a map as argument to function I know that seq is used for sequences (lists, vectors...)


Hello everyone! I am playing around with clojure (and cljs) and I just found out thing (i guess the only thing) i don't like in clojure (actually, in lisp and languages without dot syntax): It's too uncomfortable to write as you think. Let me explain on example: 1. I have names 2. Now I sort them: names.sort 3. And take first: names.sort.first I type synchronously with my thoughts. In Lisp (or some procedural languages as C) I should move cursor back and add another function invocation with closing bracket (oh my god!). How did you deal with it? I am using emacs, btw (if it helps). Sorry for my english


well that's how you're used to working with it. but you want the first sorted name. (first (sorted names))


"write as you think" is pretty language specific - when I use languages that aren't lisps I have the problem that I can't write as I think


but yes, using -> and ->> lets you put things in the order you are more familiar with


@schmee hey, thank you! Actually, I knew about this feature. But is it common to use it everywhere?


no, using it everywhere is a bad idea


if you aren't willing to learn to write lisp, there are many great languages out there that aren't lisps - but I encourage actually taking the time to learn it


it is very common and idiomatic


@dpsutton well, I guess my mind is completely different, because I often find myself in situation when I am typing sort before first.


@schmee are you telling me that a majority of your functions contain threading macros?


no, I’m saying you should not be afraid to use them


if it makes your code more readable


perhaps I took "everywhere" too literally - I've read really bad code that overuses -> and ->>


@noisesmith I played with Common Lisp back in school. Actually I very like the syntax (homoiconicity) but something was wrong. And today I found out what exactly. So I joined this channel to ask other people about it


Well, I guess I should learn to think before typing:) Thank you for your help, guys


my example was just saying what you want to do. i want the first name in a sorted list of names. so then it would be natural to describe it as (first (sorted names)). but i agree i will often think the other way as well


but the macros (-> names sort first) can make it easier to do as well


Yeah, threading macro is nice idea. I guess I just need to use it more often


also, don't be afraid to use let blocks instead of deeply wrapping / long chains of operations