Clojurians Log v2

Hey @cmal what do you intend to do? For example, I use re-frame to build an SPA, which means that my backend only serve the files & provide a storage API. express.js would serve a REST API in my case.

hi

Do you have a demo repo? I'm afraid I cannot express my work very well, I want to first dive into your demo repo and have a further description of what I need for it. Thank you!

I do the WECHAT authorization process in the server side and want to use express.js to serve my static files and serve the template html files. I also want to have the template can receive some parameters from server side.

Good so both are independant

If I where you I'd follow one of the hundreds tutorials on express to setup my backend, you'll probably write a small REST API that a client can send request to

Then look into reframe and cljs-http this template should be a good start: https://github.com/Day8/re-frame-template

OK, Thank you. I use re-frame-template but I do not know how to let the app.js generated by re-frame-template project serve as a template for express.js to use.

You're going to use templating to pass data to your user

use express.js to template your index.html

and in your index.html generate some stuff like <script>my_app_value = {{ templated_string_xxxx}}</script>

when you're clojurescript code loads, use the data from there, in a var like js/my_app_value. Don't compile with advanced optimizations.

That's a hacky solution, but it's simple and you'll come back later to fix it.

@lsenta you'd just need externs for /** @const */ var my_app_value; to keep advanced optimizations

or access via (js/goog.object.get js/window "my_app_value")

Or use oops

Or learn about the new system that'll deprecate externs soon apparently

Or.... blaaarg, just keep it simple and do it when you need it 🙂

@vendieng has joined the channel

yeah true ... just don't discard :advanced, always try to keep it working step by step

it is important and should not be left to the last minute

#clojurescript I have a piece of code that I expect to throw, so I use (is (thrown? js/Error code)). doo, keeps telling me the test fail because that's not the correct error. So I do the following: run the code, catch js/Error and throw it again in a thrown? test.

(deftest test-assoc-throws-invalid-paths
  (go-async-timeoutable S 20000
    (try
      (c/assoc-in! @db [prefix "core" @test-id "throwable/should/fail"] {:k 234})
      (catch js/Error e
        (println "EEE" e)
        (is (thrown? js/Error (throw e)))))))

Testing konserve-firebase.core-test

EEE #object[Error Error: slashes in token t=throwable/should/fail]

FAIL in (test-assoc-throws-invalid-paths) (:)
Got exception: #error {:message "Error: slashes in token t=throwable/should/fail", :data {}, :cause #object[Error Error: slashes in token t=throwable/should/fail]}
expected: (not (instance? js/Error x))
actual: (not (not true))

I literally catch the error as a js/Error to have cljs.test tells me it's not a js/Error, how is that possible?

(is (thrown? js/Error e)) not (throw e)

eh wait

I throw it again to demonstrate the problem

ah yeah the issue is core.async

it rewrites your code to SSA which looks something like

SSA?

well no idea really with thrown?

https://en.wikipedia.org/wiki/Static_single_assignment_form

not sure actually. the thrown? macro should still produce a valid try/catch

can't you just do that test sync?

Thank you very much. @lsenta

@ines has joined the channel

@dani.petrili has joined the channel

@avogis has joined the channel

@elvira_ has joined the channel

@karoline.bliemegger has joined the channel

@nessaguedes has joined the channel

@delyana.bg has joined the channel

@lena has joined the channel

@shabcat has joined the channel

@mariapurri has joined the channel

@evithan83 has joined the channel

@mels has joined the channel

@petronella has joined the channel

@tanya_lytovchenko has joined the channel

@malvacyren has joined the channel

@josefine.nordling has joined the channel

Nah, it relies on async feature, this one needs the timeout and all of my other tests will rely on an async lib.

@raphaela has joined the channel

I'm developing three apps at the same item, in ClojureScript, using cljsbuild and I'm putting common parts in a common library. What's the best way to streamline the apps picking up those changes?

I mean, changes to the common library.

I lein install, which is simple & slow

@lsenta I need to streamline the dev cycle. Slow is not ok. I also found that lein install is not good enough, the other projects wouldn't pick it up. The only thing I found that really works is to deploy the library and let one of the other projects install it, but that's even slower.

@pupeno what do you mean they don't pick it up?

@lsenta I mean, they don't have the new code. I create a function in the common library, run lein install and the function is not present on the projects that are using the library.

you do restart the projects after the install?

@lsenta first, that's even slower, but yes, I tried cleaning and recompiling and restarting and nothing.

- there are a few lib that allows injecting & dealing with jar updates at runtime, not sure what's they're worth - the non-update maybe a cache problem, I know that clojars won't update library without a -SNAPSHOT to avoid issues - have you thought of rsync'ing the files between projects? that's hackish but once you get your cljsbuild auto, rsync will be hard to beat

you could play with the :source-paths to inject your other libraries

I'd rather not have hackish thing. I would even have to figure out how to get rsync to work on Windows.

I tried playing with :source-paths and I'm not sure if I'm doing something wrong or cljsbuild refuses to read files outside its project.

@pupeno I know nothing about cljsbuild but you could try using lein checkouts and then adding that checkouts/the-lib/src to :source-paths

@thheller lein checkouts require the library to be in the checkouts directory while I need to share the library with three projects.

I thought that was the point yes? I do that frequently

@thheller the point? I'm confused.

~code/the-lib ~/code/project-a ~/code/project-b

project-a/checkouts/the-lib there the-lib is a symlink to ~/code/the-lib

@thheller there are no symlinks on windows.

as far as I know.

ah doh

hmm there is if you use ubuntu on windows 😉

I'm pretty sure there are

https://www.howtogeek.com/howto/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/

not sure if they have the same properties for lein

They are not the horrible .lnk file? interesting.

I have been playing with ubuntu/bash on windows recently and can really recommend it at this point

still a few rough edges but overall very seamless

not sure you'd be interested in that

I'm developing a desktop app for Windows and Mac; I don't want to workaround my target operating system.

yeah I understand, I meant just for the development experience

I'm happy to use Ubuntu for Windows when it's appropriate... although I found it doesn't fix the performance issues I had with Ruby on Windows.

The file system is still an issue. Accessing NTFS from Ubunut is slow and accesing the Ubuntu filesystem from Windows is impossible (unless you consider read-only-write-destroys access an option).

I have a ~/code in ubuntu which is just a symlink to /mnt/c/code

so in windows I work in C:\code and linux ~/code .. works perfectly fine and fast

@thheller that path doesn't sound like Ubuntu for Windows.

Well, Linux for Windows actually.

hmm? which path?

/mnt/c/code

That sounds like an Ubuntu VM.

nope? windows provides that (maybe only in the latest "creator" update?)

I must be misremembering then.

At any rate, if ~/code is a symlink to /mnt/c/code, then your code is on NTFS, which has performance issues (for what I tried to do, not Clojure related).

dunno I can only speak for the latest creator update and everything seemed fast there

building a large CLJS project surely didn't run into any issues

Cool.

and that writes heavily with caching and all

The issue I had was not read/write but one of the syscalls that asks for metadata about a file.

Rails has auto-code loading so it needs to see which files changed and which ones didn't and one of the syscalls that gets you the last time of change or something like that, on Windows, is horribly slow.

I run my devtools on that which watches all source paths and didn't have any issues

granted the watch has a 2sec built-in delay but that is not window-related but jvm related (also on mac)

I'm sure I posted somewhere my findings but I cannot find it now.

It's not really that windows is slower but that it doesn't provide hooks to detect file changes. Or maybe it does now.

I'm sure you tested much more extensively than I did ... I have not used ubuntu on windows in anger yet

maybe the windows links work as well

I'm experimenting with pointing source-paths... it seems figwheel sees the files but cljsbuild doesn't (trying to confirm this).

@nadejde has joined the channel

Ha! I think I found the answer.

"../clientcommon/src" doesn't work, but "..\\clientcommont\\src" does.

#notsureifbugorfeature

Documented for posterity: https://pupeno.com/2017/05/06/how-to-work-with-a-private-library-in-clojurescript/

odd that src/app works but ../clientcommon/src doesn't

@thheller tell me about it!

@kamituel has joined the channel

@sammy has joined the channel

Using cljs-http, how do I download a large file? (That is, a POST request where the browser pops up a dialog box & saves the file without reading the whole file into memory.)

(More details: my cljs client calls a Clojure service with a Ring handler whose body is a java.io.File.)

@tjg you can't really do that with ajax requests

you can hack something together using <form ...> tag and submitting that

or POST with cljs-http and have the server give you a url which you can load in the browser

@thheller Thanks! Yeah, that last solution looks good. Will do…

@ellaadem has joined the channel

@christiane2624 has joined the channel

@montsant has joined the channel

@sarahkokisawesome has joined the channel

@oskar132 has joined the channel

in clojurescript, using core.async, what would be the best way when you have a sequence of promise chans, to wait until all of them are completed ?

i’m still trying to wrap my head a bit around the subset of core.async that clojurescript supports 🙂

@lmergen wouldn't it suffice to read from them all in a row?

since each one would park until realized, and then return immediately if realized

hmmm i need to combine all the results, like a zip operation… perhaps a merge and a reduce ?

that would work, so would a normal reduce, where the last arg is the list of promise-chans, and the reducing function reads from them and uses the result

reading can only be done inside a go block, right… which returns a channel itself ?

you would reduce inside a go block - anything reading from channels will be in a go block

ok, i’m going to try this approach, thanks!

oh wait - I think you need merge and async/reduce

I forgot that the function inside the reduce loses the go context

yeah 🙂

i kind of ran into that

merge fails because promises don't close

yeah i kind of worked around that

+user=> (let [chans (repeatedly 10 >/promise-chan)] (doseq [c chans] (>/>!! c 42)) (>/<!! (>/go (>/<! (>/reduce + 0 (>/merge (map #(>/take 1 %) chans)))))))
420

where clojure.core.async :as >

much cleaner with multiple lines and a ->> macro

but that uses <!! right, which doesn’t exist in clojurescript

only for repl friendliness

its on the outer edge of the code

(doesn't look like "outer" because it's a one liner, but it is)

fairly straightforward translation to ->>

(let [chans (repeatedly 10 >/promise-chan)]                                     
  (doseq [c chans]                                                              
    (>/>!! c 42))                                                               
  (>/<!!                                                                        
   (>/go                                                                        
    (->> chans                                                                  
         (map #(>/take 1 %))                                                    
         (>/merge)                                                              
         (>/reduce + 0)                                                         
         (>/<!)))))  

this one works but looks crazy

(let [chans (repeatedly 10 >/promise-chan)]                                     
  (doseq [c chans]                                                              
    (>/>!! c 42))                                                               
  (->> chans                                                                    
       (map #(>/take 1 %))                                                      
       (>/merge)                                                                
       (>/reduce + 0)                                                           
       (>/<!)                                                                   
       (>/go)                                                                   
       (>/<!!)))

ahh!

right

I think there's a way to do this with pipeline and the take transducer

it's an interesting problem - there feels like there should be a higher level form that abstracts this pattern of collection / channel usage, but I don't see one that works

alts! works well for such things

@thheller but all the promise chans all have data available once realized

so it doesn't fit

you need to do the map of take 1 somewhere, and to me that's the messy part

hmmm, what’s the reason a promise chan doesn’t close once it’s fulfilled ?

because it's meant to infinitely return the same result every time it is read

maybe promise chan isn't the thing you want?

aren't you supposed to close! it? (never used promise-chan)

but alts! still works just don't pass in the chan once you read something from it

no - it keeps returning the same result even after close

Clojure 1.9.0-alpha15
+user=> (require '[clojure.core.async :as >])
nil
+user=> (def c (>/promise-chan))
#'user/c
+user=> (>/put! c 42)
true
+user=> (>/<!! c)
42
+user=> (>/<!! c)
42
+user=> (>/close! c)
nil
+user=> (>/<!! c)
42

chans keep returning data if buffered even after closed, promise-chans continue to have data buffered forever

oh, looping with alts and removing the read channel would work, but imho it would be even messier than what we had already

(explicit looping logic and all)

so the promise chan is actually making all this more difficult, otherwise it was just a matter of merge and reduce

right

i can work around that

(defn wait-for-all [chans]
  (go (loop [chans-to-read
             (set/map-invert chans)
             result
             {}]

        (if (empty? chans-to-read)
          result
          (let [[v ch] (alts! (keys chans-to-read))]
            (recur
              (dissoc chans-to-read ch)
              (assoc result (get chans-to-read ch) v))
            )))))

(let [foo (async/promise-chan)
      bar (async/promise-chan)]

  (>!! foo :foo)
  (>!! bar :bar)

  (prn (<!! (wait-for-all
              {:foo foo
               :bar bar}))))

man, clojurescript sure makes flaws in your async logic more obvious

could use another collection type, doesn't have to be a map

you can’t get away with using <!! 🙂

@lmergen it doesn't need <!! just easier to test in the REPL 😉

no i was talking about clojurescript 🙂

@thheller yeah, that's about what I expected it to look like - it is a lot more code, and much more imperative, than the map / merge / reduce

@noisesmith you put the wait-for-all function somewhere and never look at it again

but wait-for-all is the part I don't like

wait-for-all looks a lot like a barrier…

seems like that is the missing construct 🙂

@thheller my criterion here is the points in tim baldridge's talk at the last clojure west, removing state and io from core.async code by using higher order functions

@lmergen that is only a basic example. actual code wants to handle timeouts and such

there is no easy general solution for that

also what do you do if one chan fails

as soon as you want to account for failures a general wait-for-all doesn't work

@snamellit has joined the channel

@beckiwordsworth has joined the channel

i'm trying to figure out a good way to strip HTML received via AJAX. is there a lib out there that i can use?

@awatt has joined the channel

@alain_picard has joined the channel

Hi #clojurescript what's the correct way to have private API keys as env variable, that can be imported in a env.cljs? As far as I understand, that's a little peculiar since the evaluation should happen at compile time

@lsenta I have a half finished blog post about this

gotta go to dinner

Another option is to use Google Closure defines https://www.martinklepsch.org/posts/parameterizing-clojurescript-builds.html

my solution: https://github.com/binaryage/env-config usage: https://github.com/binaryage/cljs-devtools/blob/559054de8d2f60af0dfee6d1cc315ad5ed175a9d/src/lib/devtools/prefs.cljs#L6-L9 https://github.com/binaryage/cljs-devtools/blob/559054de8d2f60af0dfee6d1cc315ad5ed175a9d/src/lib/devtools/prefs.clj#L28

one reason is that you have to use type hints to make DCE work properly with closure-defines: https://github.com/binaryage/cljs-devtools/releases/tag/v0.5.3

(if ^boolean goog/DEBUG (devtools/install!))

without ^boolean it won’t properly DCE it

Yeah, I've been using a (onlydev (foo) (install!)) macro for years. Easier than the constant type hinting IMO.

@danielcompton you can provide them to :closure-defines from the environment, no? :closure-defines {:app.core/production (System/getenv "PROD")}… lets you avoid some macros at least

I tend to use macros, which are more predictable IMO. If I decide not to emit some code, I’m 100% sure it won’t appear in :advanced build.

but both solutions are fine, just with :closure-defines one has to be more careful

e.g. this was another issue (not really a problem of :closure-defines, but related): https://github.com/binaryage/cljs-devtools/commit/4e368408986bf739b56aad64e064b854fd7989d2

yeah, in the light of stuff like this I think I understand why you favor macros

I like the macro approach because, if you understand macros, you can read the source and understand exactly what is happening. If you are a newcomer and haven’t fully grokked macros yet the opposite may be true.

these days I tend to merge: 1) some kind of default config, 2) :external-config passed into cljs compiler options 3) and env config, at least for library code

and emit result into cljs as data

something like this: https://github.com/binaryage/cljs-oops/blob/908b6108ce5f9a823a639c6ec9293bfc3276c2bd/src/lib/oops/config.clj#L62-L68

not that one can implement nice validation on top using clojure.spec 🙂

https://github.com/binaryage/cljs-oops/blob/908b6108ce5f9a823a639c6ec9293bfc3276c2bd/src/lib/oops/config.clj#L56

here is the default config + specs: https://github.com/binaryage/cljs-oops/blob/908b6108ce5f9a823a639c6ec9293bfc3276c2bd/src/lib/oops/defaults.clj

Hum, thanks for all the links, I'm still trying to wrap my head around how you get your env variable into the clojurescript code

It seem's to me that all your example shows code that relies on having the variable already in your project.clj

@lsenta this macro clojure function reads env vars (basically what environ library does): https://github.com/binaryage/cljs-devtools/blob/559054de8d2f60af0dfee6d1cc315ad5ed175a9d/src/lib/devtools/prefs.clj#L22-L25

Ok, you rely on the macro step being called from java

which has access to the env as any jvm process

yes

here is the macro:

https://github.com/binaryage/cljs-devtools/blob/559054de8d2f60af0dfee6d1cc315ad5ed175a9d/src/lib/devtools/prefs.clj#L32

that '~ is important, to treat everything as data

and here I emit it in cljs: https://github.com/binaryage/cljs-devtools/blob/559054de8d2f60af0dfee6d1cc315ad5ed175a9d/src/lib/devtools/prefs.cljs#L8

ignore the delay call, that is there to allow DCE

for curious, here is the explanation why it was needed, see point #1 here: https://github.com/binaryage/cljs-devtools/issues/37#issuecomment-293575471

Could I get away with something as simple as

(defmacro get-env [name & [default]]
  (let [val (or (System/getenv name) default)]
    `'~val))

Almost, you don’t need either of the quotes on val since you will just return it as a literal

nor the unquote

(defmacro get-env [name & [default]]
  (or (System/getenv name) default))

Haha I've been overthinking the symbol transformation, thanks @noonian

No problem

Thanks for the details & answers, I'll stick with the straightforward solution for now

Glad to know you guys have badass solutions but secretly hoping to never need it 🙂

Hi, are there any examples that uses cljs/re-frame + express.js?