Fork me on GitHub
#beginners
<
2018-08-04
>
Daniel Hines05:08:35

I have a histogram generated by frequencies on two maps. I need to compare the two histograms, e.g assert that for every key, the value from the first map is larger than the second map, or some such. What's the best way to do that? Here's what I had at the REPL:

clojure
(every? #(apply >= %1) (map #(map second (vector %1 %2)) (frequencies (seq "aaaa")) (frequencies (seq "aaaaaaa"))))
But there must be a prettier way of doing that. Suggestions?

Bobbi Towers05:08:31

Well for one thing, you can get rid of seq

Bobbi Towers05:08:52

(strings are already seqs)

seancorfield05:08:18

@porkostomus You mean frequencies calls seq on its argument, right?

seancorfield05:08:25

String are not seqs.

seancorfield05:08:29

They are seqable.

seancorfield05:08:53

user=> (seq? "aaa")
false

seancorfield05:08:48

One of the issues for you @d4hines is that the order of keys in those two frequencies calls is not guaranteed to be the same so calling map on the two results isn't going to give you a matching set of key/value pairs.

seancorfield05:08:10

Also, are you always guaranteed that the set of keys in both frequencies results is the same? What would happen if one or the other didn't have a particular key?

dpsutton05:08:16

(merge-with >= (frequencies [:a :a :b]) (frequencies [:a :b])) if the constraint sean is asking about holds

seancorfield05:08:38

@dpsutton Dude, you beat me to it! 🙂 I was just about to suggest merge-with >= 🙂

dpsutton05:08:51

then do what you want on the vals. like (every? identity (vals ..)

dpsutton05:08:16

haha i was trying to type it fast because it felt clever

seancorfield05:08:19

(every? true? (vals ...))

seancorfield05:08:28

Yeah, or identity given that we know >= will produce true or false...

dpsutton05:08:48

i always forget about true? is all. it's clearly more appropriate here

Daniel Hines05:08:54

This is part of an anagram generator I'm working on. I'm trying to compare strings a and b so as to know if string b has all the characters and enough of them such that a could be a partial anagram. So you brought up a great point @seancorfield, the order of the keys would mess things up.

Daniel Hines05:08:34

How does this look:

clojure
(defn contains-letters2?
  [subword word]
  (let [histogram-word #(frequencies (seq %))
        hword (histogram-word word)
        hsubword (histogram-word subword)]
    (->> hsubword
         (reduce-kv (fn [m k v] (assoc m k [v (get hword k 0)])) {} )
         vals
         (every? #(apply <= %)))))

seancorfield05:08:36

So you'd also need a check on the sets of keys in both frequency results (that one is a superset of the other).

seancorfield05:08:22

But merge-with is (arguably) cleaner than that reduce etc I think?

seancorfield06:08:38

clojure.set/difference will help with the check on the sets of keys

dpsutton06:08:42

also the keys of the frequencies map being equal will tell you if you are in the same set of letters

dpsutton06:08:54

of the merge-with <= map talked about above

Daniel Hines06:08:47

The code above still addresses missing keys with the 3rd argument of the get, right? That is, if a key/char is in the subword but not in the word, the get will return a 0, the <= will cause the function to return false, right?

Daniel Hines06:08:57

Just making sure I'm not missing something.

seancorfield06:08:18

That's an interesting point... because if the set of keys is different, merge-with will produce a mix of numbers and true/false so the final check would be a bit more complicated...

seancorfield06:08:23

user=> (merge-with >= (frequencies "hello world!") (frequencies "hi lord!"))
{\space true, \! true, \d true, \e 1, \h true, \i 1, \l true, \o true, \r true, \w 1}
user=>

seancorfield06:08:29

in this case, getting numbers in the response doesn't tell you which side has extra keys -- set/difference would be needed too

Bobbi Towers06:08:03

This will check if the key is there and if not, use 0:

(defn contains-letters?
  "Returns true if letters of str2 are contained in str1."
  [str1 str2]
  (every? #(>= (if (contains? (frequencies str1) %)
                 (get (frequencies str1) %)
                 0)
               (get (frequencies str2) %))
          (keys (frequencies str2))))

dpsutton06:08:10

(defn partial-anagram? [word subword]
  (let [f-hist (frequencies word)
        sub-hist (frequencies subword)]
    (and (set/subset? (set (keys sub-hist)) (set (keys f-hist)))
         (every? identity (merge-with >= sub-hist f-hist)))))

seancorfield06:08:11

user=> (set/difference (set (keys (frequencies "hello world!"))) (set (keys (frequencies "hi lord!"))))
#{\e \w}
user=> (set/difference (set (keys (frequencies "hi lord!"))) (set (keys (frequencies "hello world!"))))
#{\i}
user=>

dpsutton06:08:43

make sure its a subset and then check identity which will pass for both true and integer values

seancorfield06:08:37

(shows I haven't used clojure.set enough -- yes, subset? 🙂 )

Daniel Hines06:08:20

Think there is any performance benefit to doing the subset? check?

Daniel Hines06:08:47

(As opposed to @porkostomus's method)

dpsutton06:08:28

i take clarity until performance is a problem. and it's not obvious that one is fast than the other without profiling. so i would do the subset and merge strategy

4
Daniel Hines06:08:34

Thanks a bunch, guys!

dpsutton06:08:09

(source set/subset?) shows they are quite similar although set creation should be faster than the histogram. (but you already have that for free). but it has an optimization based on size as well

Chase16:08:09

Hello! I was hoping to get advice on my learning programming/Clojure path. I'm a book learner but also want to start my own projects. I'm enjoying learning about the terminal and command line and would like to explore building my own command line apps as my first programming projects. I'm also a writer and I've been mulling over an art project that is a combination hybrid of an interactive website, interactive fiction, and text based browser game. I think I want to emulate terminal behavior on the web. Something like this game, UI, wise: http://web.mit.edu/mprat/Public/web/Terminus/Web/main.html That was apparently written in Java (https://github.com/mprat/Terminus/tree/master/Java/src) so I assume I could do the same with Clojure(script)? Before I ramble too much, what should I be looking for in terms of learning material? Parsing? Browser terminal emulators? It all seems too much to handle at once so looking for guidance on how you would break this down to the smallest project to explore these interests. I hope this makes sense!

sundarj16:08:13

rather than trying to get all these different concerns in one project, i'd make several different projects to learn different concerns. e.g. to start with the text bit, i'd make a basic text adventure game; for the terminal bit, i'd build just a basic terminal; ...

sundarj16:08:01

Programming Clojure builds up to writing a hangman game, so perhaps that's a good place to start.

sundarj16:08:49

Clojure Applied is another good practical Clojure book, though it's not about making a game i don't think

Chase16:08:56

Good stuff! I had those books on my reading list so that is great. I also was looking at Web Development with Clojure. Any experience with that?

Chase16:08:47

I actually watched the first video of parens of the dead a few days ago! And it was way over my head at this point. Haha. Really cool stuff though. His grasp of Clojure and Emacs is quite impressive.

sundarj16:08:23

haha fair enough! yeah he's some kind of wizard

Chase16:08:05

When you say build a basic terminal, can you elaborate a bit? I think I found the right search terms in "building cli apps with clojure" so I'm looking through some of those results now.

sundarj16:08:58

i mean, just build something small that lets you run commands like ls and cat. taking input, evaluating it, and then outputting the results. i guess it could be some kind of REPL thinking about it.

sundarj16:08:10

it's pretty easy to build your own REPLs with Clojure

Chase16:08:23

But I guess I should be looking for a "browser based terminal emulator" in clojure you think?

Chase16:08:31

Ahhh, ok. I gotcha

sundarj16:08:06

yeah i would stay away from the browser for now, just improve your knowledge with basic terminal stuff

☝️ 4
sundarj16:08:40

the browser adds a fair amount of complexity, so you really want to know your stuff before attempting to use any of the javascript terminal emulators, or writing your own

sundarj17:08:15

i'd master doing this stuff on the desktop first, then move onto the web

sundarj17:08:04

i'm a web developer myself, so maybe i'm jaded, but that's what i recommend 😛

Chase17:08:27

I understand. It was already a little overwhelming to think about as is.

Chase17:08:41

so I think I will focus on making small cli apps and/or text based games in the terminal for now. Thanks for the convo!

8
Chase17:08:06

That is funny because I'm actually in Chapter 4 of that book right now. That game is for Chapter 5. haha I haven't looked ahead yet. I've actually had to backtrack a couple times for understanding. It's slow going for me it seems but it's all good!

manutter5117:08:31

Sounds like a project I started in, gosh 2014? I’ve been meaning to get back to it, guess I’m slow. Anyway, it’s not a tutorial or anyting, but you might be interested. https://github.com/manutter51/haunted-hideout

felbit17:08:00

I started my Clojure path with "Web Development with Clojure" from Dmitri Sotnikov. It does not contain a game, but gives a good overview about the web stack in Clojure.

Chase17:08:02

That was the book I was looking at too. I think it's 3rd or 4th in line for me at this point but maybe I'll move it up. And I bookmarked that game manutter51!

Chase17:08:52

It's good to know I have a strong learning path ahead of me. I had already told myself this is a multi-year (rest of life) learning process so it's good to know there are so many good Clojure resources to guide me through it. And the community!

4
Bobbi Towers18:08:10

My first Clojure program was a Minesweeper game: https://github.com/porkostomus/tsweep

pez16:08:10

Terminus looks absolutely awesome!

pez16:08:21

@chase-lambert: You might find Land of LISP a good read since it teaches LISP by building a text based adventure game.

bartuka18:08:29

hi people, I am doing a web scraper and I found a lot of libraries related to that in clj ecosystem

henrik19:08:48

No library in particular, but in general, you don't have to care about the age of a library in Clojure. Once a particular problem is solved in a particular domain, that's it. No need to keep committing, unless the problem or the domain changes.

bartuka19:08:57

🙂 cool. I am learning by practice so I am still thinking so much like other prog languages

hoang16:08:39

I've also started learning clojure and also play with etaoin for web scraping for fun, seems to be the lastest tool

bartuka18:08:51

several of them without a commit since 2~3 years

bartuka18:08:07

which one do you recommend?

bringe19:08:32

Is there a way to shutdown a process on another thread created by a future? If I run this code, I see it continually printing in my repl, and the only way I know to stop it is to make print-number-and-wait throw an exception. Can I stop that thread another way? Will closing the repl stop it?

bringe19:08:17

I read about shutdown-agents but that doesn't kill running threads

andy.fingerhut19:08:50

Clojure has a function future-cancel that should do the job

andy.fingerhut19:08:14

As long as you have kept a reference to the future you want to cancel.

andy.fingerhut19:08:34

If the thread you want to stop is not the one created directly by the call to future, but yet another Java thread created within the code you passed to future to execute, then future-cancel will not stop that thread.

bringe19:08:02

Oh I see. So if I don't have a reference to it, how would I stop it? Would closing the repl stop it, or would it still be running somewhere in my system? And if so, is there a specific process I can kill?

bringe19:08:15

I know this isn't desirable, but just curious as to what's actually happening

andy.fingerhut19:08:53

Depending on how you started the REPL, closing the REPL can also cause the JVM process to exit. If you stop the JVM process, then any and all threads running inside of that JVM will definitely stop.

👍 4
andy.fingerhut19:08:27

I say "depending on how you started the REPL", because it is reasonably common for developers to create a REPL to an existing running JVM process, or more than one such REPL, because they want to interact with the Clojure code and/or data available there. In that case, closing those REPLs typically leaves the JVM process still running (and that is what such developers often want in that case).

andy.fingerhut19:08:13

If you start a REPL using Leiningen with "lein repl", closing that REPL session will cause the JVM process to exit, too (in unusual cases, there can be delays before the JVM process exits, e.g. see http://clojuredocs.org/clojure.core/future for such a scenario).

sb22:08:11

I would like to create an unix socket implementation. curl -s -XPOST -d' {params here} --unix-socket /../../smthg.sock http:/path .. I didn’t find exact solution. What is best? eg. I found at java com.kohlschutter.junixsocket .. but what is the experience?

andy.fingerhut22:08:17

Perhaps you have already checked, but at least except for the use of a Unix socket in place of a network TCP connection, cli-http seems to be able to do what you want. Perhaps it has an option buried in there somewhere to use a Unix socket instead?

andy.fingerhut22:08:46

If you search for "socket" in this conversation log from a couple of years ago, another person who probably knows more about cli-http than I do mentions the concept of "interceptors", that can be used to customize the behavior of cli-http. It sounds like at that time the behavior you want wasn't in cli-http itself without such a customization: https://clojurians-log.clojureverse.org/clojure/2016-11-28

👍 4
sb00:08:39

Thanks the detailed answer. Tomorrow I will check all opportunities 👍👍

sb05:08:18

Thanks that is simple clj-http req.. :face_with_rolling_eyes: you dont need any extra thing..