Fork me on GitHub
#beginners
<
2017-10-31
>
drewverlee00:10:59

i would have expected ["foo" ""] from (str/split "foo/" #"/") maybe im used to how python does this.

sundarj01:10:29

fwiw, this does what you want:

(clojure.string/split "foo/" #"/" -1)
["foo" ""]

drewverlee00:10:51

I find it odd i can't figure out a way to verify is a string is a valid path where i could write a to.

drewverlee00:10:35

Like, it seems your best bet is to try to write the file and catch the exception.

phronmophobic00:10:15

the java File object has .exists and .isDirectory that you can use

drewverlee00:10:50

Yea. But i want to know if the program can write to a location. its getting kind of tricky. "/" => nope "A" => yep "valid/path/foo" => yep "invalid/path/foo" => nope "valid/path/" => nope (no file name)

noisesmith00:10:59

@drewverlee maybe something like this?

+user=> (defn valid-file? [f] (let [file ( f)] (or (.isDirectory file) (.exists (.getParentFile file)))))
#'user/valid-file?
+user=> (valid-file? "/etc/passwd")
true
+user=> (valid-file? "/etc/passwd-not-on-disk")
true
+user=> (valid-file? "/etc-no-such-dir/passwd-not-on-disk")
false

noisesmith00:10:52

oh, now that you clarify, make it (and (not (.isDirectory file)) (.exists (.getParentFile file)))

noisesmith00:10:56

oh - that still doesn't work for "A"

phronmophobic00:10:08

although, there might be other reasons you can’t write to the file

phronmophobic00:10:18

like out of space or permissions

drewverlee00:10:16

Yea. I agree, checking everything would be somewhat involved and mean a lot of trade offs. I was just curious about the basic stuff concerning file paths

drewverlee00:10:44

This is what i have so far:

(s/def ::valid-file?
  (s/and
   #(not (str/blank? %))
   (s/or
    :can-write               #(.canWrite (io/file %))
    :in-current-dir          #(nil? (.getParentFile (io/file %)))
    :valid-file-path         (s/and
                              #(.isDirectory (.getParentFile (io/file %)))
                              #(not (str/blank? (last (str/split % #"/"))))))))

drewverlee00:10:11

the last condition doesn't wok because of how string split works.

noisesmith00:10:52

it looks like this one meets your spec

+user=> (defn valid-file? [f] (let [file ( f)] (and (not (.isDirectory file)) (.exists (.getParentFile (.getCanonicalFile file))))))
#'user/valid-file?
+user=> (valid-file? "A")
true
+user=> (valid-file? "A/B")
false
+user=> (valid-file? "project.clj")
true

drewverlee01:10:34

Thanks! Ill admit i felt like getCanonicalFile might have been the missing piece to making this easy, but i couldn't figure out what it did. Thane name and docs seem to assume some background i dont have.

noisesmith01:10:20

the canonicalFile is absolute and has a parent (even if it's implicit in the original) while the raw file is dependent on your pwd

phronmophobic00:10:09

i don’t know a lot about your use case, but it seems like that spec definition might cause headaches later

phronmophobic00:10:37

my intuition is that if some data conforms to a spec, it should always conform to the spec

noisesmith00:10:59

oh - yeah, that's highly contingent isn't it

phronmophobic00:10:25

and not depend on things like what’s on the filesystem where the spec code is being run

drewverlee00:10:05

Why would that limitation be helpful.

drewverlee00:10:43

It will make that spec somewhat meaningless for generators, i admit.

phronmophobic00:10:34

it makes the spec function not pure anymore

phronmophobic00:10:05

which means that if you, for example, use spec to check conformance on a build server and it fails

phronmophobic00:10:37

then you don’t know if it’s a problem with the data you’re checking or if there’s some setup problem on your build server

phronmophobic00:10:26

in other words, it means that any code that uses that spec now has the implicit argument of the filesystem

phronmophobic00:10:50

and it’s generally harder to test implicit arguments

drewverlee00:10:12

Sure. But the thing im trying to verify depends on the state of the filesystem.

drewverlee00:10:11

My plan was to verify inputs with it, so that i could give better feedback to the user.

phronmophobic00:10:23

yea, it does seem like a useful thing to specify, that your function expects a writable file path

phronmophobic00:10:50

or, as another example, your function expects an active socket connection or something

phronmophobic00:10:54

ok, nevermind. seems like a useful thing to do

phronmophobic00:10:09

i wonder if there’s a convention for saying your spec has implicit arguments

drewverlee00:10:20

I'm glad you brought that up, its worth thinking about.

phronmophobic00:10:20

like ::valid-file?!

rcustodio01:10:15

is protobuf a good choice for clojrue or not? is anyone using it? https://github.com/ghaskins/clojure-protobuf

seancorfield01:10:51

@rcustodio What problem are you trying to solve that you think protobuf might be (part of) the solution for?

rcustodio01:10:35

Communication over rmq / tcp

seancorfield01:10:39

Between arbitrary processes? Or between Clojure processes?

noisesmith01:10:47

I've found transit very useful - it's faster than edn, and supports all the clojure data types, and the customization for extra types is first class and local instead of implicit and global

rcustodio01:10:33

I guess it will be between clojure processes only, idk if the project will adopt others Lang in the future

rcustodio01:10:05

Just the api and websocket is json, but that is easy to converti

seancorfield01:10:39

If you're doing Clojure-to-Clojure for now, I'd stick to EDN for the first version, then switch to Transit and see if it improves performance etc.

seancorfield01:10:00

For EDN, there are other language readers/writers. I assume there are for Transit too?

rcustodio01:10:18

What is this transit? Is it possible to pass it through rmq?

seancorfield01:10:35

It's a pass-through-able as Protobuf 🙂

rcustodio01:10:48

I know few about edn as well

rcustodio01:10:01

I just know the edn file

rcustodio01:10:19

Hmm, i will research about it

seancorfield01:10:12

"Transit is supported in Ruby, Python, Clojure, JavaScript, Java."

rcustodio01:10:29

Thanks ✌️:skin-tone-2:✌️:skin-tone-2:✌️:skin-tone-2:

noisesmith01:10:20

yeah, transit might have better multi language support than edn, and it's definitely better performing and more flexible compared to using pr-str and custom reader tags or whatever

rcustodio01:10:36

And is better and more able to use in clojure than protobuf?

noisesmith01:10:54

a lot easier to use, at least

rcustodio01:10:06

I see, thanks

Michael Stokley04:10:03

is it good practice to set up infinite "go" processes to listen to channels?

Michael Stokley04:10:13

e. g. (go (while true (some-func (<! input-channel)))

Michael Stokley04:10:35

i tried (close! channel) but it doesn't seem to help.

noisesmith04:10:52

a closed channel won't interrupt a loop that reads from it - if you read from a closed channel it instantly returns nil, and the consumer should be checking for that

Michael Stokley04:10:42

cool, i can do that

noisesmith04:10:51

(go (when-some [v (<! input-channel)] (some-func v) (recur)))

Michael Stokley04:10:49

i'm traversing a graph asynchronously. i need a way of keeping track of visited nodes. would an atom be a suitable ref type for that?

noisesmith04:10:03

that seems reasonable, yeah - just be aware that if you hit conflicts (two or more threads trying to alter the value at once), you will end up seeing retries. You'll want to make sure that the inevitable retries don't break things with your core.async usage, and if performance matters at all you probably want to do batching of some sort, or limit the recursion branches somehow

Michael Stokley04:10:18

i think it should be because i don't need - i don't need any kind of transaction logic

noisesmith04:10:43

with something like graph traversal, the retries can easily be more expensive than the navigation is, making the parallel version slower than the single threaded synchronous one

noisesmith04:10:11

it's faster to look up a value in a hash and put it in a set, than it is to look up a value in a hash, create a mutation checkpoint, attempt to put the value in a hash, check the checkpoint, and either retry or return the new value

noisesmith04:10:58

but I don't know what your data looks like or how it is acquired, only testing will tell you what performs best

Michael Stokley04:10:30

it's urls. it's a scraping project.

noisesmith04:10:33

@michael740 specifically the big problem is with parallelizing something like that is that the lookups of nodes in the graph is so much faster than the mutating transactions that the larger the number of threads, the larger of your percentage of time is spent on automatic retries, so the slower the whole thing gets

noisesmith04:10:54

OK - with URLs that's different - try different parallelisms to see what has the best throughput though

Michael Stokley04:10:57

so i send a request, parse the html for more links to fetch, go fetch them in parallel, ...

noisesmith04:10:38

also you could check out memoize - if you memoize the "get links from URL" function that might simplify some of your logic and eliminate retries

noisesmith04:10:08

but then again there's no predicate to see if a memoize has seen an arg already, so that's probably a bad choice

Michael Stokley04:10:54

if i have a get-links-from-url function and i've seen the url already, i think i just want to move on.

noisesmith04:10:26

right, that's fair - and I checked and memoize is just doing a swap! anyway

Michael Stokley04:10:21

i appreciate your help!

Hendrik Poernama08:10:52

is there an 'identity' transducer? I'm thinking (map identity) if nothing else.

leonoel09:10:38

the identity transducer is identity

Dima13:10:28

hi, im new to programming, but i want to start from clj, plz give me some advice where to start ?

chris13:10:15

clojure for the brave and true

Dima13:10:05

i heard about it, is this the most understandable way to start?

chris13:10:44

I think it's likely to be the most approachable way to begin programming

chris13:10:04

SICP is a great book, but likely to be less approachable unless you're into math

chris13:10:41

I didn't realize that was the SICP class, I thought it was the book

amarjeet13:10:58

yeah, its the videos, and has course assignments 🙂

Dima13:10:51

thnx, other thoughts are welcome )

themistoklik13:10:20

if you go with sicp

themistoklik13:10:35

please take reaaaaaally small bites, especially if you're new to programming

themistoklik13:10:05

make sure you get what you're reading

Dima13:10:22

i found & really interesting, thnx!

sundarj13:10:40

there are more in the series, by the way

Dima14:10:38

cool, 10x

rcustodio13:10:36

@seancorfield and @noisesmith I’m checking out about transit to communication, now to validate data is schema a good choice? (https://github.com/plumatic/schema)

ghadi14:10:56

I would look at clojure.spec instead

seancorfield15:10:03

Yes, definitely look at clojure.spec first as that is "built-in" to Clojure 1.9.

rcustodio15:10:11

Since I’m still developing my app, it will be out march or later, is it wise to use clojure 1.9?

ghadi16:10:03

It's essentially in release candidate status now

rcustodio16:10:40

So probably it will be final until then?

seancorfield18:10:24

Alex said they plan no changes between Beta 4 and "gold", except to fix any regression bugs introduced.

rcustodio18:10:11

Great, so I will start to use it in my new projects

seancorfield19:10:56

We're using Beta 3 in production, with Beta 4 in testing right now.

seancorfield19:10:19

(we've had 1.9 in production since one of the first Alpha builds)

rcustodio19:10:44

Woow, okey, thanks that helps a lot

dpsutton13:10:47

if you're going to do sicp, there's a really great online version http://sarabander.github.io/sicp/html/index.xhtml

Joe Lane17:10:00

Hey friends, I’m aware of destructuring with keys which allows me to write

(defn foo [{:keys [foo bar baz]} some-map]
    (println foo bar baz))
I was wondering if the inverse of destructuring, (restructuring?) exists or if I’m just googleing the wrong name or if this is even a good idea at all. I’m thinking it would look like this
user> (let [foo "foo"         ;; This let is just used to create lexically scoped values
        bar "my-bar"
        baz [1 2 3]]
(->map {:keys [foo bar baz]})) ;; or some other syntax, I didn't put a ton of thought into it.
=> {:foo "foo" :bar "my-bar" :baz [1 2 3]}
Am I totally missing something?

noisesmith17:10:18

I have a gist for that

Joe Lane17:10:50

thanks @noisesmith, it looks like it captures all locals, which isn’t quite what I was looking for but this looks super cool.

noisesmith17:10:05

it’s easy to combine it with select-keys

noisesmith17:10:33

(or just write a macro that expands to a map with a key for each symbol)

Joe Lane17:10:29

great, I see what you’re saying now. Thanks @noisesmith

noisesmith17:10:22

(ins)user=> (defmacro restructure [syms] (into {} (for [sym syms] [(keyword (name sym)) sym])))
#'user/restructure
(ins)user=> (def a 0)
#'user/a
(ins)user=> (def b 1)
#'user/b
(ins)user=> (let [c 2] (restructure [a b c]))
{:a 0, :b 1, :c 2}

noisesmith17:10:51

it’s only so often you can write a macro with no quoting, heh

Joe Lane17:10:26

Wow. It’s pretty cool to see you work through this stuff.

lepistane20:10:02

what's idiomatic way to name a map as argument to function I know that seq is used for sequences (lists, vectors...)

Nondv21:10:00

Hello everyone! I am playing around with clojure (and cljs) and I just found out thing (i guess the only thing) i don't like in clojure (actually, in lisp and languages without dot syntax): It's too uncomfortable to write as you think. Let me explain on example: 1. I have names 2. Now I sort them: names.sort 3. And take first: names.sort.first I type synchronously with my thoughts. In Lisp (or some procedural languages as C) I should move cursor back and add another function invocation with closing bracket (oh my god!). How did you deal with it? I am using emacs, btw (if it helps). Sorry for my english

dpsutton21:10:01

well that's how you're used to working with it. but you want the first sorted name. (first (sorted names))

noisesmith21:10:06

"write as you think" is pretty language specific - when I use languages that aren't lisps I have the problem that I can't write as I think

noisesmith21:10:49

but yes, using -> and ->> lets you put things in the order you are more familiar with

Nondv21:10:24

@schmee hey, thank you! Actually, I knew about this feature. But is it common to use it everywhere?

noisesmith21:10:54

no, using it everywhere is a bad idea

noisesmith21:10:35

if you aren't willing to learn to write lisp, there are many great languages out there that aren't lisps - but I encourage actually taking the time to learn it

schmee21:10:39

it is very common and idiomatic

Nondv21:10:44

@dpsutton well, I guess my mind is completely different, because I often find myself in situation when I am typing sort before first.

noisesmith21:10:19

@schmee are you telling me that a majority of your functions contain threading macros?

schmee21:10:38

no, I’m saying you should not be afraid to use them

schmee21:10:49

if it makes your code more readable

noisesmith21:10:03

perhaps I took "everywhere" too literally - I've read really bad code that overuses -> and ->>

Nondv21:10:04

@noisesmith I played with Common Lisp back in school. Actually I very like the syntax (homoiconicity) but something was wrong. And today I found out what exactly. So I joined this channel to ask other people about it

Nondv21:10:54

Well, I guess I should learn to think before typing:) Thank you for your help, guys

dpsutton21:10:55

my example was just saying what you want to do. i want the first name in a sorted list of names. so then it would be natural to describe it as (first (sorted names)). but i agree i will often think the other way as well

dpsutton21:10:11

but the macros (-> names sort first) can make it easier to do as well

Nondv21:10:57

Yeah, threading macro is nice idea. I guess I just need to use it more often

noisesmith21:10:48

also, don't be afraid to use let blocks instead of deeply wrapping / long chains of operations