Fork me on GitHub
#clojure
<
2020-09-07
>
Eddie01:09:08

Is there a good pattern for passing context created in a test fixture into a the test? For example, if we have a fixture that creates a new directory named with (System/currentTimeMillis) how would we make the directory's path available within a test? My current best attempt is to use an atom and avoid any parallel test running. That works fine, but I am curious to hear if anyone else has a better approach.

Eddie01:09:23

The two recent instances where I was thinking about this are: 1. The per test "scratch" directory I mentioned and 2. Creating a new local Spark warehouse/metadata store after each test.

seancorfield01:09:19

First off, I'd recommend using a proper temp directory instead of try to make the name up yourself. I'd probably use a ^:dynamic var and have a binding inside the test fixture. That should respect threads so I think it would solve the parallel problem too?

Eddie02:09:05

Thanks @seancorfield. Can you elaborate on "proper temp directory"?

Eddie02:09:51

Gotcha, thanks! Definitely seems like a smarter way to go.

seancorfield02:09:29

Yup. No need to reinvent something the JDK already provides.

kenny02:09:20

@erp12 Most of our tests create stateful contexts (e.g., db connection, file system, etc) using with-open in the body for a deftest. Has worked out real nice, completely replacing fixtures.

Eddie02:09:25

Good idea@kenny. I have used with-open in tests for reading specific assets, but the reason I originally didn't reach for it in this instance is because I don't want to have to know about the thing I am opening. If I am understanding java.nio.File/createTempDirectory (per @seancorfield's suggestion) correctly, I should be able to write a little boilerplate so that with-open will work again like:

(with-open [test-dir (create-temp-dir)
  ...)
Thanks all!

quadron02:09:31

is there a lib out there for comparing hashmaps? say i want to talk about hashmaps kind of like how set theorists talk about sets. spec kind of does this, but I was thinking like something that is immanent to the hashmap structure. for example, how do you express this: hashmap1 is a subset of hashmap2

Eddie02:09:22

@veix.q5 Check out the functions in clojure.data for example: https://clojuredocs.org/clojure.data/diff

✔️ 3
quadron02:09:01

@erp12 thanks, that's useful

datran03:09:51

Is there a tool that will create a graphiz chart of namespace dependencies?

datran13:09:52

this works to show the relationships between your namespaces and your deps, but I want something to show the graph of my own namespaces, if that makes sense

Charles Comstock21:09:49

I've used https://github.com/greglook/lein-hiera for this in the past, though I think last time it wasn't working as I expected. Still looking for the equivalent tool to use with clj though, keep meaning to translate it.

datran14:09:05

This worked really well, thanks!

mrchance13:09:52

Hey, I found http://clojuredocs.org/clojure.core/partition-all#example-542692d4c026201cdc32702a on Clojuredocs, which is buggy and confused a coworker 😄 What's the best way to fix that? Should I just edit it inline providing an explanation?

stathissideris14:09:09

in clojure/tools/[email protected] how do you pass a string as an argument? I’m getting:

» clj -X:convert :repo-path "/tmp"
Unreadable arg: "/tmp"

stathissideris14:09:31

thanks 🙂 a bit awkward I think

vlaaad17:09:35

> a bit awkward wait until you see quoting in powerhell 😄

vlaaad17:09:59

""""/tmp"""" ...

😃 3
stathissideris14:09:19

It seems to be trying to read it as a symbol

kingcode15:09:59

Is there a channel for cognitect’s Rebel tool, and/or discussion of datafy/nav? I looked for #rebel channel, but no result came back. It exists, because I couldn’t create it.

lispyclouds16:09:30

#rebl might be what you're looking for?

seancorfield17:09:18

Cognitect's tool is REBL, but there's also a rebel readline project (not Cognitect): https://github.com/bhauman/rebel-readline (in case that's where the confusion comes from?).

kingcode03:09:29

Thanks! Got the spelling wrong..

noisesmith16:09:01

somehow I didn't think of this until today

(require (.name *ns*) :reload)

👏 3
svt17:09:25

I’m playing around creating and deploy clojure lib to clojars this is the url of my code : https://github.com/cksharma11/clo-utils When I’m trying to use this library in my test project I’m getting error. Could not locate clo_utils/core__init.class, clo_utils/core.clj or clo_utils/core.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

dpsutton17:09:30

Is your test project a lein project? I saw on your read me that you expect people to use this as a plug-in and that seems wrong. This should just be a regular dependency

svt17:09:41

Oh sorry that’s my bad. It’s wrong in readme. I’ll correct it, But I’m using it as a dependency in my test project

dpsutton17:09:33

/tmp ❯❯❯ bat deps.edn
───────┬─────────────────────────────────────────────────────────────────────────────────────────
       │ File: deps.edn
───────┼─────────────────────────────────────────────────────────────────────────────────────────
   1   │ {:deps {clo-utils {:mvn/version "0.2.0-SNAPSHOT"}}}
───────┴─────────────────────────────────────────────────────────────────────────────────────────
/tmp ❯❯❯ clj
Clojure 1.10.1
user=> (require 'clo-utils.core)
nil
user=> (dir clo-utils.core)
if-empty
if-nil
if=
when-empty
when-nil
when-not-empty
when-not-nil
nil
user=>

dpsutton17:09:39

seems to work for me

dpsutton17:09:44

how are you using it?

svt17:09:51

(defproject test "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url ""
  :license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
            :url ""}
  :dependencies [[org.clojure/clojure "1.10.0"]
                 [clo-utils "0.2.0-SNAPSHOT"]]
  :repl-options {:init-ns test.core})

svt17:09:19

(ns test.core
  (:require [clo-utils.core :refer :all] ))

(defn foo []
  (if-empty [] 1 2))

svt17:09:20

core file

svt17:09:40

@U11BV7MTK Hey seems to work for me too.

svt17:09:31

I’m not sure what was the issue but seems to work fine now.

dpsutton17:09:36

awesome. did you not restart your repl maybe?

svt17:09:40

Thank you

jaide18:09:04

Does anyone have an example of using migratus with deps.edn on hand?

p-himik18:09:47

I just run the relevant migratus.core functions in a -main function of some file that I load in an alias. With the new clj that supports executing arbitrary commands this should be easier.

jaide18:09:23

Something like clj -m my.migrate.wrapper :migrate?

p-himik18:09:13

Yep, exactly.

p-himik18:09:37

Although that -m part is in an alias, so more like clj -Amigrations migrate.

jaide18:09:19

Thanks! This is what I have so far. Last thing I need to figure out is how to provide my existing postgres db connection via jdbc driver from the app to migratus. Feels weird having to specify it twice.

p-himik18:09:46

My DB connection URI is written in config.edn that's read by both the app and the migration script. I also use Integrant, but having a common config makes sense even without it.

p-himik18:09:31

Also, instead of that huge function registry, you can just use

(let [fn (resolve (symbol (str "migratus.core/" op)))]
  (apply fn config args))

jaide18:09:27

I’m following https://github.com/Folcon/fulcro-template/commit/edb1c099825983fe38bbe29293c51a3880db8ddc to connect the fulcro app template to a postgres instance. I’ve got the config in my config/dev.edn and it uses mount to create a database connection pool so in theory I would expect to be able to point migratus to that.

jaide18:09:36

That’s a great improvement. Thanks again!

👍 3
seancorfield19:09:17

With the new Clojure CLI -X option, I suspect you could streamline it even further, since you could put your config in deps.edn as an alias, and you could directly invoke any migratus.core function that took a single hash map argument -- so you'd only need wrappers for up and down.

p-himik19:09:25

How would that look up config? Note that it can be quite complex.

seancorfield19:09:28

@U2FRKM4TW The :exec-args structure can be an arbitrary hash map, and you can override it on the command-line. I haven't looked in any depth at how Migratus handles configuration but I'm basing this on the code that @U8WFYMFRU showed.

seancorfield19:09:07

If I get around to trying Migratus some day, that's probably the approach I'd take.

p-himik19:09:38

Yeah, I get what you're saying. But how can you actually extract a map from within a common config file? This is the only thing I can come up with, and it doesn't look pleasant.

clj -X migratus.core/migrate <(clj -e ...some-code-to-read-and-extrac-config...)

seancorfield19:09:08

I'd put the config directly in deps.edn and drive Migratus from that.

p-himik19:09:23

Oohh, that does sound interesting, thanks.

seancorfield19:09:25

:aliases
{:migratus {:extra-deps {...}
            :ns-default migratus.core
            :exec-args {:store                :database
              :migration-dir        "migrations/"
              :init-script          "init.sql" ;script should be located in the :migration-dir path
              ;defaults to true, some databases do not support
              ;schema initialization in a transaction
              :init-in-transaction? false
              :migration-table-name "foo_bar"
              :db {:classname   "org.h2.Driver"
                   :subprotocol "h2"
                   :subname     "site.db"}}
...
}
and then clojure -X:migratus init which will invoke migratus.core/init and pass that config in.

seancorfield19:09:50

(untested but I suspect it'll work based on the README for Migratus)

seancorfield19:09:18

You'd need a wrapper for up/`down` because they take two args -- a hash map and a migration ID -- but your wrapper could be

(defn up [config+] (migratus/up (dissoc config+ :id) (:id config+)))
and then you'd say clojure -X:migratus my-wrapper/up :id '"20111206154000"'

seancorfield19:09:47

You can override any part of the :exec-args, even nested parts, via the command line so you only need your defaults in there.

seancorfield19:09:05

clojure -X:migratus <migratus-function> :init-script "local.sql" '[:db :subprotocol]' '"h2:mem"'

seancorfield19:09:43

(I'd prefer to use the {:dbtype "..." :dbname "..."} format for :db since it's easier to work with)

jaide20:09:52

That -X feature looks awesome! However, I think I prefer having a single wrapper that supports all the commands vs some being in the config and some as wrappers. Though I can definitely use that feature for other tools\projects though.

jaide23:09:26

Got it working if anyone wants it for reference.

Vishal Gautam20:09:54

Hello. I am trying to do write my schema to a file system. This is the function I have.

(s/def :entity/id
  (s/with-gen
   uuid?
   #(s/gen uuid-samples)))

(s/def :entity/type #{:entity.type/user :entity.type/card :entity.type/topic})

(defn gen-initial-schema []
  (spectomic/datomic-schema
   [[:entity/id    {:db/unique :db.unique/identity
                    :db/doc "ID of the entity"}]
    [:entity/type {:db/doc "Type of entity"}]]))

(spit "resources/datomic/schema.edn" 
      (pr-str {:app/norm1 {:txes [(gen-initial-schema)]}}))
When I try to run the function below. The namespaces gets messed up in the edn file. Like so
#:app{:norm1 {:txes [[#:db{:ident :entity/id,
Whats the best approach deal with this. Thank you 🙂

cgrand20:09:53

It’s just *print-namespace-maps* being set to true.

💯 3
cgrand20:09:37

However, as @seancorfield said, it’s valid EDN.

Vishal Gautam20:09:58

@U3E46Q1DG Thank you so much. It solved my problem 😄

Vishal Gautam20:09:14

@U3E46Q1DG another question. How do you make sure that the data saved is formatted properly. I tried using pprint from clojure.pprint but it returns nil so didnt work

cgrand20:09:50

Wrap it inside with-out-str.

seancorfield20:09:39

@vishal.gautam What do you mean "messed up"? That looks like valid EDN... Have you tried reading it back in? What failures do you get?

Vishal Gautam20:09:15

Sorry on the poor choice of words. I mean the file was saving keys in namespaced keywords, like we normally would write

seancorfield20:09:17

I'm still not sure what you mean by "messed up". #:foo{:bar 42} is the same as {:foo/bar 42} The #:foo part is a reader shorthand for "all the keys in this map are qualified with foo/"

😮 3
🤯 3
seancorfield20:09:25

user=> {:foo/bar 42}
#:foo{:bar 42}
user=> {:foo/bar 42 :foo/a "A"}
#:foo{:bar 42, :a "A"}
user=> #:foo{:bar 13 :a "X" :b "Bee"}
#:foo{:bar 13, :a "X", :b "Bee"}
user=> (keys *1)
(:foo/bar :foo/a :foo/b)
user=> 

Vishal Gautam20:09:50

@seancorfield I was not aware of that, thank you 😄

hiredman00:09:54

The end spec such that it is(https://github.com/edn-format/edn/blob/master/README.md) doesn't mention namespace maps, so they may infact not be valid edn (clojure is a superset of edn, a number of features of the clojure reader are not part of edn)

seancorfield00:09:51

edn/read-string accepts them:

user=> (edn/read-string "#:foo{:a 1}")
#:foo{:a 1}
user=> (keys *1)
(:foo/a)
user=> (edn/read-string "#:foo{:a 1 :b/c 2 :d 3}")
{:foo/a 1, :b/c 2, :foo/d 3}
user=> (keys *1)
(:foo/a :b/c :foo/d)
user=>