Fork me on GitHub
#clojure
<
2019-12-01
>
markxnelson01:12:37

Hi, I am trying to write a test for this function:

(defn find-copyright-statements-in-file
  "Find copyright statements in the given file"
  [file]
  (let [re #".*[Cc]opyright.*(19|20)[0-9][0-9].*"]
    (with-open [r (io/reader file)]
      (remove nil? (map first (map (partial re-find re) (line-seq r)))))))
I have this in my test:
(deftest test-find-copyright-statements-in-file
  (let [output (c/find-copyright-statements-in-file file)]
    (is (= (count output) 1))
    (is (= (first output) expected))))
This gives me an error:
ERROR in (test-find-copyright-statements-in-file) (BufferedReader.java:122)
expected: (= (count output) 1)
  actual: .IOException: Stream closed
 at .BufferedReader.ensureOpen (BufferedReader.java:122)
    .BufferedReader.readLine (BufferedReader.java:317)
    .BufferedReader.readLine (BufferedReader.java:389)
I was playing around in the REPL... I found that if I do (def x (find-copyright-statements-in-file some-file)) then I can do things like x, (count x), (first x) and they work fine, but if I try to do it in a let instead of a def I get these same errors - e.g.
copyright=> 
(let [y (find-copyright-statements-in-file file)]
  (print (count y))
  (print (first y)))
Execution error (IOException) at .BufferedReader/ensureOpen (BufferedReader.java:122).
Stream closed
class .IOException
I do not understand why it works in the def but not in the let. I assume it has something to do with then the lazy seq is actually evaluated... any help appreciated! Thanks in advance

markxnelson01:12:40

Oh - answering my own question - I guess typing it out suggested the answer.... I change the function to this

(defn find-copyright-statements-in-file
  "Find copyright statements in the given file"
  [file]
  (let [re #".*[Cc]opyright.*(19|20)[0-9][0-9].*"]
    (with-open [r (io/reader file)]
      (doall (remove nil? (map first (map (partial re-find re) (line-seq r))))))))
Now it works like I expected 🙂

duckie 12
AlesH20:12:25

BTW I hink you can replace (remove nil? (map ...) with keep Also take a look at transducers. You can put together all the mapping functions by calling them without the collection argument, then you can use that xf with into instead of doall, because it is eager when used with a transducer. It may give you some performance gain.

andy.fingerhut02:12:39

Without the doall, the rest of the functions you are using there are lazy, which prevents them from being evaluated within the scope of the with-open, and if you try to force the lazy sequence after leaving the scope of the with-open, the file is closed and gives an error.

💯 4
p4ulcristian12:12:02

Hello there! Anyone using Datomic in production? How much space will it accumulate over time? I mean, we don’t lose data, which is a pretty fancy cool thing, but what about when I don’t have more space? Are there some functions to reset the database every month for example. So the state one month ago will be the “first state” of the database?

dominicm12:12:31

#datomic is the place for this

dominicm12:12:09

But you can decant the database into a new one

dominicm12:12:51

If you're planning on putting that much data into datomic, you might struggle. There's a soft limit on the data you can put in, and you'll need to speak with cognitect for capacity planning.

Nima G14:12:25

Is Gradle the way to go if I want to write both Java and Clojure in a project?

jsyrjala14:12:23

I would first look at Maven for that.

val_waeselynck14:12:23

Also consider Leiningen, in particular in combination with https://github.com/ztellman/virgil

8
4
Nima G14:12:31

Thanks for the replies. I actually didn't know about java-source-paths in leiningen

alexbaranosky17:12:58

Yeah. If you're already using leiningen, I'd just go that route ^^^

👍 4
borkdude15:12:49

When AOT-ing a project with JDK 11 and deploying it to clojars, will the project be able to run with JDK 8? I may have accidentally done that

jsyrjala15:12:48

Whether a .class file works on certain JDK version depends on the bytecode version that .class files have. https://clojure.org/community/devchangelog says that clojure 1.10 generates Java 8 bytecode.

borkdude15:12:37

ah, so if I don't use any fancy java 11 features, chances are it still works?

jsyrjala16:12:43

It won’t work if you used parts of JDK libraries that are in JDK 9 or later. Otherwise it could work.

borkdude15:12:02

and is there a way to check which java version has been used for a specific artifact?

Alex Miller (Clojure team)17:12:11

If you have class files, javap can tell you which version they are

borkdude19:12:40

bug or feature?

(defn var
  ([name] (var name nil nil)) ;;=> always returns #'clojure.core/name
  ([name init-val] (var name init-val nil ))
  ([name init-val meta] (SciVar. init-val name meta)))
Probably using var as a function name in a public API (or at all) is not a very good idea

didibus20:12:50

I don't think special forms can be shadowed

didibus20:12:21

Or it seems they can, but not if they're the first element of a form

borkdude20:12:18

I had a similar issue recently

didibus20:12:30

I don't know if that's correct or not, but I like to think of them as read time macros

didibus20:12:02

Like they get evaluated at read time, even before macro-expansion

didibus20:12:46

Hum... well ya that doesn't seem to be the case. But oh well. I'm not too sure what the semantics are for special forms and when they are processed exactly. But I'd avoid shadowing them 😝

andy.fingerhut20:12:01

It seems like a thing a linter one might want to warn about 🙂

❤️ 4
andy.fingerhut20:12:16

I don't think Eastwood does so.

borkdude20:12:29

hm yeah.. 🙂

didibus20:12:59

I think they're processed directly by the compiler. And depending which special form it is, they'll have the compiler do something different. But in the context where you shadowed a special form. I guess you could consider it a bug that the compiler misidentifies the code as a special form when it isn't referring to one. At the same time, I don't know if it's possible (or just way too hard) to detect the difference.

borkdude20:12:02

I have this inside a clojure interpreter:

;; derived from (keys (. clojure.lang.Compiler specials))
;; (& monitor-exit case* try reify* finally loop* do letfn* if clojure.core/import* new deftype* let* fn* recur set! . var quote catch throw monitor-enter def)
(def special-syms '#{try finally do if new recur quote catch throw def . var set!})

andy.fingerhut20:12:19

I mean, just imagine the confusion that one could create by trying to name a function .

borkdude20:12:20

so I guess the same will happen when you call a function throw

andy.fingerhut20:12:08

I had fun debugging Eastwood running on a Clojure library that defined, I believe, a macro named catch

didibus20:12:12

(def def (fn [] "lol"))

andy.fingerhut20:12:29

which I believe worked as the authors intended it to.

borkdude20:12:05

user=> (def throw [])
#'user/throw
user=> (throw)
Syntax error compiling throw at (REPL:1:1).
Too few arguments to throw, throw expects a single Throwable instance

didibus20:12:39

I think it does work

borkdude20:12:50

user=> (defn catch [])
#'user/catch
user=> (catch)
nil

didibus20:12:54

Just need to fully qualify your fn name namespace/var

borkdude20:12:01

catch probably works because that symbol only has a special meaning inside throw

andy.fingerhut20:12:13

something like that, yeah.

borkdude20:12:41

@U0K064KQV it could work with a namespace prefix, but what if the function is recursive like in my original post?

borkdude20:12:53

aaaanyway, good to lint

didibus20:12:05

`(def def (fn [] "lol"))
(def) ; too few args to def
(def a 10) ; #'a
a ; 10
(user/def) ; lol`

andy.fingerhut20:12:27

It does not come up very often in live code examples, but there might be something out there.

borkdude20:12:21

oh yes, this works:

user=> (defn var ([x] (user/var x nil)) ([x y] [x y]))
#'user/var
user=> (user/var 1)
[1 nil]

borkdude20:12:01

still probably good to avoid. I bet I will trip some edge case in CLJS or something when I go ahead with this

didibus20:12:07

Ya. I also don't really understand the underlying mechanism. I guess the compiler probably just first checks if the symbol in first position is a special form, if so treats it as one. If not, performs var resolution in the given namespace. And when the symbol is fully qualified, it does not equal any special form so that probably works.

didibus20:12:25

Seems to work in CLJS as well. But ya, who knows what other edge case it could have.

sogaiu21:12:53

i don't know if the details are still accurate but there is a bit in the "clojure for java programmers" (part 1) rich hickey talk that mentions about the order -- around 1:03:00 or so (the op slide)

andy.fingerhut22:12:06

Yes, although that might be a somewhat simplified explanation vs. what the compiler actually does, even at the time that talk was given.

andy.fingerhut22:12:33

The details of name conflicts can get pretty finicky here, I suspect.

sogaiu22:12:21

i hope to some day understand it well :)

andy.fingerhut22:12:36

Eh, if you get insanely curious, I would recommend adding several System.out.println calls into likely-looking places in the Clojure compiler implementation, then come up with several example forms to compile and evaluate that should exhibit different behaviors in that area. It can be time consuming, but on the order of hours, not weeks.

andy.fingerhut22:12:10

well, perhaps hours stretching into days 🙂

sogaiu22:12:51

thanks for the tips -- may be my curiosity level will reach that point before long

bja19:12:12

anyone know of any efforts to use Clojure as a macroassembler for systemverilog?

bja19:12:59

it's something I've been thinking of quite a bit recently, given past experience using Clojure for DSLs/compilers and a new work focus on SV

theeternalpulse22:12:17

is it bad practice to shadow a variable via a let binding that's out of scope?

theeternalpulse22:12:28

granted not a core clojure function

andy.fingerhut22:12:07

Not sure what you mean by "that's out of scope" there. Do you have an example of what you mean?

andy.fingerhut22:12:06

I mean, how can you shadow something unless it is visible in the scope of where the let expression is?

theeternalpulse22:12:50

for example, I often use key destructuring, but if I need to process a key before passing it thoruhg, I often shadow

(let [{:keys [a b c]} some-obj
      a (if (= b 2) a 0)],,,)

theeternalpulse22:12:04

sorry, you're right, not out of scope

andy.fingerhut22:12:04

In a small enough let, I would think the meaning is pretty clear to all all slightly experienced Clojure developers.

andy.fingerhut22:12:28

The larger the let gets, the harder everything is to follow, not just shadowing.

💯 4
theeternalpulse22:12:49

I often do a' or something, but evil-cleverparens has some annoyances with stray single-quotes

andy.fingerhut23:12:31

I am not familiar with evil-cleverparens, but if that is intended to be a Clojure or Lisp-specific mode, it might be advisable to report that and see if they can improve on it.

dpsutton22:12:29

I think so.

rutledgepaulv23:12:11

i wouldn’t read into that too much. may just mean many clojurians are happily employed, effective in small numbers, and anti-hype 😛

didibus23:12:42

Don't know what to say... Clojure is amazing, but the battle for mindshare is tough, and Clojure puts little focus on it and doesn't really care. Or should I say, wouldn't compromise its goals to make itself more attractive, for good reasons.

Alex Miller (Clojure team)23:12:41

Those are based on very small percentage differences at one online job site

Alex Miller (Clojure team)23:12:17

My impression is that there are more companies hiring more people now than ever

Alex Miller (Clojure team)23:12:22

Was just at Clojure Conj, full of people at companies hiring Clojure devs

💯 24
ajs23:12:06

That's good to hear.

herald09:12:38

I wonder what's the threshold for being included on that list. I can imagine the job postings mentioning clojure went from 5 to 1, just by chance of the measuring date.

tbiernacki11:12:15

It's kind of weird to see Clojure (a relatively new and popular technology) among other dinosaurs like jsp, solaris (!), vba, flash, windows XP (!!) ...

tbiernacki11:12:55

I don't necessarily pay too much attention to such reports but I still find it peculiar that among all possibile niche technologies that exist and are declining it's Clojure has the taken first position.

ajs11:12:26

Compared to flash it was particularly surprising for me as well. That suggests, according to the worlds largest jobhunting site, that more people are interested in sticking with flash for their work than clojure?

tbiernacki11:12:57

Maybe that's because there are few corporate jobs and companies are simply no longer posting their positions on that website anymore. Maybe most recruitment these days happens in other forms (other hiring platforms, this slack's channels, conferences ect).

tbiernacki11:12:28

But it could be as well just an anomaly in the way they processed their data. I doubt that people are sticking more with Flash than with Clojure.

tbiernacki11:12:22

Also I remember that in the past many backend -positions would list Clojure as a "nice-to-have" technology in their description. Maybe it's no longer the case nowadays and this highly influenced the final result.