@jaen: That's great! Will definitely have a look and try it out

If anyone is using Datascript, I’d be very interested to get your feedback or own experience with subscribing to db updates in a clean and quick way. While I’d like to reuse datalog queries for subscribing to updates that would affect those very same queries, I didn’t make much headway. I settled for now on filtering the transaction report datoms by hand. See this gist: https://gist.github.com/wildermuthn/390a663014d0dd4955e0

@wildermuthn: I tried an approach where multiple listeners could subscribe using core.async pub/sub to pre-filtered streams (you could listen on an entity, or on an attribute).

How can I have a namespace spanning multiple files in a folder?

@wildermuthn: turned out this wasn't such a great idea, it didn't mesh well with React, so I abandoned it.

@jrychter: Performance problems?

Lifecycle timing?

@wildermuthn: that too, but more importantly it turned out that you have to completely change the way you manage React component updates, which isn't always a good thing. The idea of rendering as much as possible from a single app-state is good. It's best to aim for it.

In terms of passing the entire state into the root, then cursors down into children?

As for DataScript, I find it extremely useful, but it can't hold all of my app state yet, because of performance.

@wildermuthn: I use Rum, so no cursors there. But essentially I try to either have components which just receive simple arguments, or react to the global app-state.

@jrychter: Gotcha. I haven’t gotten into performance problems with datascript, per se, but I don’t have much data yet. At some point, does having too many entities become just too slow?

@wildermuthn: if you start reacting to DataScript changes, you run into issues. For example, if you're not careful and pass the entire db down the component tree, every component will end up re-rendering on every db change. Not good. Entities are another trap: they are compared by their identity, and yet they can change. You won't be able to easily detect that change.

@wildermuthn: DataScript is really fast. But you do reach limits eventually — in my case, it was a large table with "selected" checkboxes in each row. Keeping the :ui/selected attribute in DataScript is a natural solution, but caused performance problems.

I think DataScript is great, but not to keep all of your app state, but as a side database. As an example, my selected rows now live in a separate Clojure set, which is way faster and works well. Which doesn't mean I don't need DataScript — if you have relational data, it's a godsend.

Right now, I only make queries to DataScript following particular ui actions (flux-like), and I convert the query entities into Reagent atoms that are just plain cljs maps.

So on subsequent queries, the Reagent atom is reset, and the component re-renders.

But it’s a mess, keeping track of when to manually re-query based on ui actions.

Which is my motivation for subscribing to db changes. It won’t work to have the components re-running their datalog query on every db update.

@wildermuthn: I wrote a macro where I pass a query and a predicate, and it returns me a javelin cell (similar to your reagent atom), which is only updated when my predicate says this change in interesting

like that:

(defmacro derived-cell
  [db id check q & args]
  `(let [inner# (tailrecursion.javelin/cell
                  (datascript/q ~q @~db ~@args))]
     (datascript/listen! ~db ~id
       (fn [tx-report#]
         (when (some ~check (:tx-data tx-report#))
           (reset! inner# (datascript/q ~q @~db ~@args)))))
     inner#))

(i still need to document it)

to be honest, usage is a bit quirky, I'm thinking about converting the macro to keyword arguments, so that they partially serve as a documentation when somebody's reading the code

because right now it looks like that:

(def *campaigns
  (data/derived-cell
    data/db
    ::campaigns
    #(= :campaign/id (:a %))
    '[:in $ ?contains-now?
      :find [(pull ?e [*]) ...]
      :where [?e :campaign/id]
             [?e :starts-at ?start]
             [?e :finishes-at ?finish]
             [(?contains-now? ?start ?finish)]]
    #(t/within? %1 %2 (t/now))))

@asolovyov: this makes a lot of sense.

@wildermuthn: I would love to piggie-back on query and check if it's needs to be re-run by using that query, but that's much harder problem and I needed something to fix 1s delays right now

@wildermuthn: in my case, it isn't the db driving the changes. It's always a UI event or a server-side push. The db "lags behind". So it doesn't make much sense to make React components depend on the database in my case. But it depends on the application.

I should've used word 'deriven' though. ;-)) Can't invent a better name...

or is it derived? 😕

@jrychter: well, sometimes you finally get the data to your request from server, and you should show it, and how do you know the time has come?

@asolovyov: what you have looks like the most straightforward way of subscribing to changes, which is to create a transaction-checking function that suits the query.

What do people use to manage cookies these days in browser-oriented CLJS applications?

@asolovyov: I’m using core.match on the entire set of tx-data datoms, like so:

I use a "dispatcher"-based architecture, with a tiny pub/sub central dispatcher. Anyone can generate an event, and listeners can register to listen on events. So anything that happens in the app is through an event, which means I know that something has changed.

@wildermuthn: but there is still need to write down the predicate, right? Or you parse the query itself?

@asolovyov: I don’t parse the query, no. I tried for the past few days, but all my experiments were dead-ends.

There’s a PR that tries to do query analysis: https://github.com/tonsky/datascript/pull/12

@jrychter: by the way, with those central dispatcher and actions based architectures, you lose any traces of typing cljs provides you, right? This thing bothers me a bit - if I'm using function directly, compiler will at least complain if a function does not exist, or a number of arguments is invalid, and in case of pubsub... you just have to check it by hands

@wildermuthn: but @tonsky said it's not exactly good, so there is that

@asolovyov: yes. I haven't found this to be a problem.

@asolovyov: yeah, and I couldn’t understand what the PR was doing very well, either.

@jrychter: like, you just check it every time and it's more or less ok?

@tonsky said, "I’m not sure that the whole idea of “reverse” query can be solved, even in datalog"

So that’d didn’t inspire a lot of confidence! hah

@asolovyov: I am not sure if I understand what you mean. You mentioned typing, and my answer was about typing. I don't rely on type checking much.

@jrychter: I wonder if core.typed can be used here to assure me or something

yeah, I don't as well

but if you do (dispatch! :action-name), and there is no :action-name, you'll never know until you execute that code

@asolovyov, @jrychter Either of you or anyone else know the folks behind http://precursorapp.com?

and if you do (some-ns/some-fn), it will show you a warning during the build process if this function does not exist

@wildermuthn: Nope. My app is PartsBox, https://partsbox.io/

@wildermuthn: not really, @sgrove does though

Ok, thx. I’ll check out PartsBox too. Precursor is pretty amazing, and the DataScript readme mentions they use it. Very interested to get their perspective.

Yes, Precursor gets impressive performance, very snappy

@jrychter: http://partsbox.io is very snappy too! I really like the interface. I see those checkboxes you mentioned, although the demo doesn’t have too many.

Ah, I found the show more link.

@wildermuthn: if you click the top checkbox (in the header), all the parts get selected. Now, the state of this checkbox is computed from the set of selected parts (it should be selected if all you see on the screen is selected, which could be limitedy by search or filtering). This is an example of something you do not want to blindly throw into DataScript.

@jrychter: yeah, even with your optimization, it took about 270 ms to render all the checks

I can imagine it would take even longer if you had to query the db

163 ms after I reloaded

@jrychter, @asolovyov Thanks for sharing your experience! These are good problems to have, dealing with powerful front-end databases 😉

React and ClojureScript are great, but they aren't magic. If you have lots of rows and you need to update every row, you will pay the price. The solution is pretty much: "don't have lots of rows visible".

And yes, these are the "programming first world problems" I am amazed we can discuss these sorts of "issues", while building large, high-performance apps quickly. ClojureScript is amazing.

@venantius: if you ask about cookies re: user auth then I suggest to use JWT tokens instead - http://funcool.github.io/buddy-sign/latest/ - and just store them in localstorage - https://funcool.github.io/hodgepodge/ - along with the rest of app state. At least that's what I do and I'm pretty happy with it.

Looks like it would be nice to have reactive datalog

@venantius: I would look at whatever Google Closure provides re: cookies

A CI question: cljsbuild outputs some JSC_DUPLICATE_PARAM errors but still ends with Successfully compiled … and gives me an exit code of 0. Any ideas why? (not the errors, I know what those are)

@grav are you using Prismatic Schema?

yes

@dnolen ^

that’s your problem

file an issue with them, people constantly report this issue and so far no one has followed up on it

dnolen: just to be sure, we’re not talking about the compilation error, right? The ‘success’ error code is because of prismatic schema?

Success is success. The warnings are because of prismatic macro stuff,

Oh, but I’ve solved the JSC_DUPLICATE_PARAM by not destructuring in the defcomponent. I just thought I’d get a non-zero exit code if something in the compilation failed.

It's not a failure. Warning only

okay

I guess I need to hook into the warnings and fail the build then

@venantius: re: routing, I’m a big fan of bidi and pushy, super nice

@jaen: Following your custom module work very closely, really excited by your prgress

@sgrove: thanks; I'm excited too, since I really wanted to have material-ui working with minimal modifications and it's looking more and more doable

Though if it sticks it'll be a while before it gets into stable releases probably.

@pupeno: I just read your blog. Interesting. Does it also work when you prerender a reagent component that fetches data via ajax and then renders itself?

@borkdude: sadly, no 😞 I was looking into that yesterday.

@pupeno: any idea why it didn't work?

borkdude: Nashorn just doesn’t implement xmlhttprequest.

@pupeno: I had that suspicion. What about using PhantomJS, etc?

Integrating with node might be the way to go, but I don’t like any of the integrations I seen so far. Implementing xmlhttprequest for nashorh is another possibility which sooner or later, I think it’ll happen.

@pupeno: last time I studied this area, I bumped into: https://github.com/prerender/prerender - looked promising

borkdude that could be a solution.

https://twitter.com/rauschma/status/643476865383014400

Hah, that's two weeks ahead of schedule. Way cool ❤️

Woot!

“schedule” :) http://www.reactnativeandroid.com/

Well, yeah. It still nice to see they made it in that time frame.

@jaen good timing for all your help on processing Node.js style CommonJS modules

Hah, right. It's all packaged that way for react native as well.

Incidentally in my sample React does work, but React devtools don't seem to recognise it; so I guess it's all internally consistent for the requires, but resulting object structure isn't exactly what would result from browserify and whatnot, so I'll have to investigate that.

But maybe I can do that with a custom module loader or something

anyone used the :warning-handlers of cljsbuild? I’d like to stop a build on warnings, and I believe this is the place to hook in, but the hook doesn’t seem to do anything

https://github.com/emezeske/lein-cljsbuild/blob/master/README.md#custom-warning-handlers

@grav those are for ClojureScript compiler warning handlers

those are unrelated to Closure warnings / errors

ah, I see. so no way of hooking into those?

would be nice in a CI context to stop the build. Currently I’m thinking of parsing the output of the compiler, but that seems like a hack

@grav we do not currently provide any hooks into the Closure warnings

yet another interesting patch for someone to work on

anyone know off hand if unused entries in a map literal get removed in closure advanced compilation?

@isak not for ClojureScript data literals - Closure doesn’t know how to reason about them

@dnolen i see, thanks

@isak if you want dead code elimination to work correctly you definitely don’t want to be putting anything in a map at the top-level

i think a normal js-obj will be good enough for my use case: https://closure-compiler.appspot.com/home#code%3D%252F%252F%2520%253D%253DClosureCompiler%253D%253D%250A%252F%252F%2520%2540compilation_level%2520ADVANCED_OPTIMIZATIONS%250A%252F%252F%2520%2540output_file_name%2520default.js%250A%252F%252F%2520%2540formatting%2520pretty_print%250A%252F%252F%2520%253D%253D%252FClosureCompiler%253D%253D%250A%250Avar%2520MYENUM%253D%2520%257B%250A%2520%2520%2520%2522FOO%2522%2520%253A%25201000%252C%250A%2520%2520%2520%2522BAR%2522%2520%253A%25201001%250A%257D%250A%250Afunction%2520hello(name)%2520%257B%250A%2520%2520alert('ENUM%253A%252C%2520'%2520%252B%2520ENUM%255B%2522BAR%2522%255D%2520)%253B%250A%257D%250Ahello('New%2520user')%253B%250A

heh

err, typo, it actually inlines it

top level JS objects are susceptible to DCE

provided the values are static

yeah that is great for my use case

@jaen I’d love to hear more about how you use those two in combination

I am in this case, as you guessed, specifically interested both in user auth and in appropriate UI/UX surfacing

in the past I’ve dealt with this by storing a user’s id and email in both a session cookie and a normal cookie

session gets read for server-side auth, normal cookie gets read for UI/UX cues

@venantius: there's not much to it - I have an endpoint which accepts user and password and returns a JWT token generated with buddy; then I put that token in an atom client-side and on each request I just send it in a header, say Api-Token or something. Then the API handler has a middleware that checks for this headers, decodes the token and puts current user in the request map.

There's not much to it besides that.

I imagine I could show you code of my university project, but it's kinda atrocious (I kinda suck at coding) and I'm afraid it would confuse you more than help you.

For the server side these blogposts are certainly a way better introduction - http://rundis.github.io/blog/tags/buddy.html

Though if you really want your soul to rot in hell eternal you can take a look at those three files: https://github.com/jaen/projekt-bd-frontend/blob/master/src/cljs/medisoft/frontend/api.cljs#L23-L37 https://github.com/jaen/projekt-bd-frontend/blob/master/src/cljs/medisoft/frontend/ui/application.cljs#L155-L165 https://github.com/jaen/projekt-bd-frontend/blob/master/src/cljs/medisoft/frontend/api/core.cljs

If anything's unclear feel free to ask

Hi, does any one know of a demo project using one of the react wrappers + relay with datomic?

@aramz not aware of anything like that

thanks

@jaen that’s really interesting, thanks for sharing

in many ways a very close parallel to the pattern I’ve been following in the past

I'll repeat the obligatory warning: if using jwt, always verify the encryption type when receiving it, all implementations are required to support the "none" method, which is about as secure as you might guess

@noisesmith: I’ve used JWTs previously at a financial institution so I’m appropriately paranoid 😉

that is, all implementation must support the "none" method, but you are of course allowed to reject tokens that use that method

OK

but I didn’t use them for anything client-side

it was purely server to server communication

that required signature validation

venantius: it kind of blew my mind when I found out about the "none" bug, and how easy it was to exploit

noisesmith: does this not allow for the bug you're mentioning or is that not enough? - https://github.com/funcool/buddy-sign/blob/master/src/buddy/sign/jws.clj#L105-L106

@jaen: that allows a client to specify :none in the header, and it would be accepted

you specifically have to whitelist the algo you accept (or blacklist "none")

so no, that's not enough to prevent the bug

if it is clearly specified in the docs that it should be implemented why is it a bug?

xifi: because jwt supposedly is a security thing, and when you specify an encoding of :none it provides zero security

Hmm, I thought this code means that you compare alg which is an option you specify to unsign, that is the algorithm you expect, with the one in header (:alg header) and throwing an error if they don't match

Which should mean that it won't accept :none unless you explicitly call it with {:alg :none}, but I may be missing something

jaen: in that scope alg is coming from the header

oh, wait

you are right, don't mind me, that's right

so yes, that is sufficient

That's good to hear, I didn't put much thought into that besides "I hope the implementer knew what he was doing" yet

So good to know he did ; d

@grav This is as far as I could take that dupe parameter error. https://github.com/Prismatic/om-tools/issues/81