I'm considering making a library for clojure that is used to generate spir-v code for targeting vulkan enabled gpus

spir-v is a binary specification

does anyone know of a simple library for generating binary data?

like byte arrays?

what sort of functionality are you looking for? generic sugary erlang syntax?

i've done some byte fiddling in the past and i shot myself in the foot with big endian / little endian byte order and i just don't want to fool around with that part of it if i can avoid it

your output format is a file?

yeah

I can't imagine not needing to know about byte ordering

😞

in any language

just because I imagine that's a fundamental aspect of the spec

it's like wanting to use json without having to worry about how the keys are spelled

point taken

I just googled "clojure erlang binary syntax sugar" and most of the results are about elixir

i guess its not a big deal

its just something i have to get right

there is gloss

it would be a fun macro exercise to do something equivalent to erlang's syntax

here is what i'll be working on: https://www.khronos.org/registry/spir-v/specs/1.1/SPIRV.html#Binary

maybe one of the libraries already does that

Gloss is based on the errand stuff I think

Erlang

https://github.com/ztellman/gloss

gloss looks pretty good

looks like what i had in mind

i'm looking at https://github.com/clojurewerkz/buffy too

Hey all, looks like with the new predicates in clojure 1.9 core.match spits out WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/boolean? when included

Is there any way I can work around this? trying to make a command line application that returns json and this gets printed everytime preventing me from piping to jq

move to a newer tools.analyzer

https://github.com/clojure/tools.analyzer/commit/6909fec3655ac194d9253191ffa9f1ecb76dc336

stick a grep -V WARNING in there

is every-kv supposed to work on vectors?

user=> (s/conform (s/every-kv integer? string?) ["x"]) :clojure.spec/invalid

it would be awesome if it did 🙂

it seems to work for empty vectors:

user=> (s/conform (s/every-kv integer? string?) []) []

filed CLJ-2080

What does explain tell you?

user=> (s/explain (s/every-kv integer? string?) ["x"]) In: [\x] val: "x" fails predicate: vector? nil

Looks like it's expecting vector entries

I'm not honestly sure of the intention

based on the doc string, i expected it to treat the vector like an associative collection

eg a map with integer keys

anyway, sorry to dump a problem and run, but dinner time!

how to predefine a function so you can use in file before it is read?

got it declare

hihi

are there plans to have vector namespace syntax similar to map namespace syntax - to better support datomic pull syntax, specifically attributes list in query?

Anyone know why ring is not setting cookies on response?

(-> (ok "Hello")
        (assoc-in [:cookies "X-WEB-AUTH-USER"] {:value "Thing", :max-age 3600})
         trace)

{:status 200, :headers {}, :body "Hello", :cookies {"X-WEB-AUTH-USER" {:value "HALO", :max-age 3600}}}

which ns does ok come from @johnnyillinois

(require '[ring.util.response :as res])

  (-> (res/response "hello")
      (assoc-in [:cookies "X-WEB-AUTH-USER"] {:value "Thing", :max-age 3600}))

>> {:status 200, :headers {}, :body "hello", :cookies {"X-WEB-AUTH-USER" {:value "Thing", :max-age 3600}}}

works fine for me

@johnnyillinois Make sure you use the ring.middleware.cookies/wrap-cookies middleware.

You can also use ring.util.response/set-cookie instead of assoc-in ....

hi

I want to get inputs from user into a list

Thanks!

and use that to pass it to a function

I have written like the following

(defn mylist[] (map parse-int (repeatedly tc read-line)))

and passed to function myfunction like this

(map myfunction mylist)

but myfunction is called

for each input from the user

instead I want to get all the input from the user

and then pass it to the function

sorry, if it is not clear

please ask me any input if required

@mak can't you just call (myfunction mylist) ?

my requiement is something like this, I want to take input from the user and pass each number to the list and confirm , if it is odd or even

i have written to print the even or odd

now I am reading the list from the user for let us say for 3 inputs

once I take the all the 3 inputs from the user then only I want to pass that list to myfunction

currently, for each user input 'myfunction' is called

myfunction takes only one parameter

so I think , we need to call like (map myfunction mylist) only

as this will take each parameter from the list and applies the function

correct me, if I am wrong

@mak: what you mean is that you need to realize the full list before applying myfunction to each element, correct ?

in this case, you can use doall, e.g (map myfunction (doall (my-list)))

btw, if you're running myfunction for side effects (e.g printing to the screen), it's preferable to use run! instead of map: (run! myfunction (doall (my-list)))

Wow! Thanks a lot @val_waeselynck

that is exactly my requirement

Now, I have an additional query

while printing the output, I am getting

(nil nil nil)

for each println

now how can I avoid printing it

It's not printing it, its the return value

There has to be one.

It does not affect the execution of your program outside of the repl

Thanks again @val_waeselynck. It's a great help to me.

I was reading Rich's interview about spec and have found the following snippet: "But, yes, I'd like to see more namespaced keys. We're adding a little bit of syntax to Clojure to make that even easier, literally just easier so that people do more of it." -- what does he refer to?

@dottedmag stuff like this http://dev.clojure.org/jira/browse/CLJ-1910

and destructure has been extended for namespaced keyword entries

@ambrosebs thank you

hey there - is there any way to use transducers without the reducing function? I have a composed set of filters, and I just want the filtered records; not reduced to a value...

Maybe use into?

@alexmiller Thanks! I just spotted that... Appreciate it

Can I test some functions that are depend on reponses of a external api and if so, can someone share some examples how I can make such a test

you mean with spec?

Does spec work on 1.8. I thought spec only works on 1.9 ?

there's spec as a lib somewhere on github

but if you have trouble creating a simple test for this, maybe don't dive in generators just yet

@mpenet thanks, I think I will read a lot about specs

I hope I can find out how to mock the api server because I think test schould never use the real api

yep, sounds like the way to go

so I can validate a json response and test if the outcome of the parsing will be what I expect. ?

I can not find any examples of this with google 😞

TIL about run!

@jcromartie ???

"Today I Learned" @val_waeselynck mentioned it in a discussion about doall etc. above

I have been using doall and map forever, and didn't know about the addition of run! to clojure.core

oke, I thought that was a answer to my question

haha, sorry

NP

@jcromartie check out dorun too

@roelofw I see a lot of questions up there 🙂

you want to mock an external service for tests?

what part are you stuck on?

How to do it

I have read a lot about spec and other test frameworks but I never see a example of this

the simplest way is probably using with-redefs

you just redefine the function that calls the external service with a mock version

for the scope of your tests

keeping in mind that with-redefs is not thread safe and updates the root binding of vars

but it's fine for tests

@roelofw you can also use something like http://www.mbtest.org/, it’s language agnostic and allows you to stand up mock http/https/smpt/tcp servers

oke, so I run the external api like this :

(client/get  "" {:as :json :query-params {:key (env :key) :format "json" :type "schilderij   

How do I redefine this ?

by making a variable with some json in it ?

that get call should be inside a function

i.e. you've got a function somewhere that calls this API, yes?

so if it's (defn foo [] (client/get ...))

yes. I showed you that

you showed me the detailed code that calls the API, so I don't assume you have it in a discrete function

here you see the whole function : https://github.com/rwobben/paintings/blob/master/src/clj/paintings2/api_get.clj

thanks

great

so in your tests you'd use (with-redefs [paintings2.api-get/read-numbers (constantly [1 2 3...])] your-test-code)

substituting whatever expected return value you want

oke, so 1 2 3 is the output of the external api ?

yeah, just as an example

right

oke, then I see it

(constantly x) creates a function that just returns x

so you're substituting your functions that call the API with functions that just return test values

and if you need more state than that you can use an atom or something and make your redef functions more complicated

oke, now choosing what for test-framework I m going to use

clojure.test

done

😛

oke, so no spec ? or midje ?

midge is nice

but if you don't know clojure.test then you should at least get familiar with it

oke, next thing on my learning path

if you ever wonder "which x should I use", and a core library is one of the options, learn the core library first

nice to experiment with it the next few days/weeks

knowing clojure.test means you can contribute to most open source Clojure libs, and using it in your own lib means most devs can contribute to yours

yep, that is why 4clojure challenges is a lot about solving with only core functions

This is a personal project. I do not thing many other clojure devs are joining my project

time for a very very late lunch. In 2 hours I can eat the evening dinner 🙂

@jcromartie thanks for the help

sure thing

chips, something not right here :

(ns paintings2.test.get-api
  (:require [clojure.test :refer :all]
            [ring.mock.request :refer :all]
            [paintings2.api-get :refer :all]))

(deftest test-app
  (testing "get numbers"
    (let [response (with-redefs [paintings2.api-get/read-numbers (constantly [{:id "1234"} {:id "abcd"} ])])
          (is (= [ "1234" "abcd"] (map (get-in response [:body :artObjects] ) response)))]))) 

I see this error message : let requires an even number of forms in binding vector in paintings2.test.get-api:8

https://clojuredocs.org/clojure.core/with-redefs

look at how to use redefs

also, i'm not sure you're testing anything here

@roelofw that's just a 'syntax' error, you misplaced the closing bracket of your let

you're just looking at the object that your redefined function returned, rather than looking at the logic you have coded

there are a few issues here

let's slow down

I try to test if the code can take the id of a json response which a external api has returned and is parsed by the get

jcromartie oke, I slow down

first: with-redefs redefines a var only within the scope of with-redefs itself, so since there's nothing in the body of with-redefs your mock function is never called.

You need to put your test code that calls the redefined functions inside the body of with-redefs

second: it looks like you're expecting your function to return something much different from what you're returning using constantly

what is :body and :artObjects ?

forget about the API call for a moment and just focus on mocking your functions

:body and :artobject are keys in the parsed response of the external api

yes

the external api sends a big json response and I only need a small part of it

so, it looks like your function read-numbers returns values parsed from that response

not the response itself

yes

so when you mock read-numbers, you are just skipping the API call entirely and returning what you expect read-numbers to return

oke, that will be something like [ "123" "456" ]

right

and roelofw, which function do you want to test here?

where the response could look like this : [ { :id 123 :painter "me"} { :id 345 :painter "you}]

the read-numbers function first

but you are mocking read-numbers, how could you test it

you aren't even calling it

@dpsutton oke , back to the clojure.test manual

you don't need to read the manual

what function do you want to test

that's step 1

if you want to test your response parsing, then mock client/get and returns the response you expect

^

oke, that is what I tried to do

Is there a nil-safe implementation of Clojure maps? Like one that throws with (get {} :a)

(get m :foo ::not-found)

the existing implementation is what I'd call "nil safe"

I can mock it up with : (with-redefs [paintings2.api-get/read-numbers (constantly [{:id "1234"} I think

Sorry I do not mock it up like this

no problem

the most important question is: what are you trying to test?

yeah. which function are you trying to test

post the body of the function you are trying to test here

let's see what its dependencies are

if read-numbers parse the json right and I see a map of only the objectNumbers

there's also the barely readable (:foo {} ::not-found)

awesome

post that function here

so this part :

(defn read-numbers
  "Reads the ids of the paintings"
  []
  (->> (client/get  "" {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}})
                 :body
                 :artObjects
                 (map :objectNumber))) 

and remember, redefs is making a fake implementation of a function. so we cannot possibly fake the function we want to test

so you want to verify that the logic within read-numbers is correct?

yep

awesome. now we are onto something

@dpsutton that was my example, initially, because I thought he wanted to test code that used read-numbers

my mistake

i was thinking the same thing

@mpenet @jcromartie sorry probably messing up terminlogy. I want a thing that throws when theres no value for a given key

that was the value of saying exactly which function we want to test

so we have to mock the client-get line up with with-redefs

^ perfect

right ?

yes

we are mocking the dependencies of the function we want to get

now, here's another suggestion

@martinklepsch just add a (when (= ::not-found ...) (throw ...)) and you get that

we might want to refactor

maybe we want to call read-numbers on the result of the client get

@martinklepsch no, clojure's associative data structure functions are rather forgiving

@martinklepsch alternatively contains?

and that way read-numbers is pure

and then we don't have to mock anything at all

oke, so we make a sort of fake respons , @dpsutton

then the call site would look like (read-numbers (client/get url options))

and that way options and url are just vars

well, we just call that function on different data and see what we get

oke, so first refractor the "real" read-numbers ?

@mpenet all this requires an extra fn/some wrapping. I want something that exposes the same interface as regular maps but throws in said case

@martinklepsch you'll just have to write it

@jcromartie haha thanks, the reason I asked is because maybe anyone knows of somebody else who already did 😄

@dpsutton : Schould I rewrite the read-numbers function ?

just refactor, not rewrite

@martinklepsch yup, or use something like potemkin and define your own map type (here be dragons)

oke, I have to think then how I can do that

pull the client/get out and just assume you have the response as the parameter

its just the first part of your threading macro

just move that to the call site of read-numbers

and you are done

oke, the call site is here :

(defn do-both-in-parallel [ids]
  (let [paint-thread (future (pmap read-data-painting ids))
        image-thread (future (pmap read-image-url ids))]
    (map merge @paint-thread @image-thread)))  

@mpenet ah neat, def-map-type seems handy didn't know potemkin has stuff like this

and that calls this one :

(defn read-data-painting
  "Reads the title, description, date , collection, colors and url of a image"
  [id]
  (let [art-objects (-> (str "" id )
                        (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}})
                        :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name)
        description (:description art-objects)
        date (get-in art-objects [:dating :year])
        collectie (first (:objectCollection art-objects))
        colors (:colors art-objects)]
    {:id id :name name :description description :date date :collectie collectie :colors colors}))

so I have to refractor a lot of functions then

because this one is also using client/get which also can be refractor to get this out

I was afraid we might just be kicking the can down the road

you don't have to refactor out every client get

the read-data-painting is great

it doesn't do very much work

it just destructures the json more or less

that's a great place to have a client/get inside of it

it reads the numbers and do a client-get to get a new response to get more data of that painting

its very much take an id and then return the response in a format that we want

its logically self contained

oke, but can I test this function later on then ?

i take an id and i return the json data in a format that you like

i like that function

let's leave that off for now

I have another function that do the same to get the image out of of 1 painting

Is there a higher order function or idiomatic way to convert a sequence of maps to a map based on a key fn? Or is it just reduce ex. (reduce #(assoc %1 (:id %2) %2) {} [{:id 1} {:id 2}]) ;;=> {1 {:id 1}, 2 {:id 2}}

so where is read-numbers called?

that's the function we want to test

@dpsutton im still missing you, the last function takes the output of read-numbers

here , (map merge @paint-thread @image-thread)))

i don't see a call to read-numbers

Wait , I see what you mean

if we pull the client/get out of read-numbers, where is the call site(s) that we would need to update

it drifted far away, but this is the code we want to test

show me the code that calls this function

that is here :

ns paintings2.routes.home
  (:require [paintings2.layout :as layout]
            [compojure.core :refer [defroutes GET  ]]
            [ring.util.http-response :as response]
            [ :as io]
            [paintings2.api-get :as api]
            [compojure.route :refer [resources]]))

(defn home-page []
  (layout/render
    "home.html" {:paintings (api/do-both-in-parallel(api/read-numbers))  }))

(defroutes home-routes
  (GET "/" [] (home-page))
  (resources "/")  )  

in the file routes/home.clj

ok. so if we pull it out, we can make it look like this

or even move the paintings threading macro into the let bindings

let's get the data, and then throw that through out pipeline

yep, I think slack messed up your indentation

but I see now the idea

cool

make read-numbers take data and return data

and then call read-numbers with that data

and then I can test it with dummy data that I can make up, right ?

yup

then you don't redefine anything

you just call your function with whatever data and make sure it spits out what you expect

@roelofw I can’t help but feel this really still belongs in #beginners ...

^

and I can do the same with the other two functions

oke, I will ask questions in the beginnners channel , I apolize

Thanks. Happy to help you in that channel.

NP

i have one question which I will ask in the beginners channel

@jrheard: what do you think about bifurcate as a name for the function formerly known as winnow?

@camdez I like that

@jcromartie Thanks for the feedback!

totally works, yeah

@jrheard: I’m trying to decide if it carries a hint of “two even parts”, but I don’t think it does. “Bisect” is also an option, but seems (to me) to have more of that “equal” implication.

Idk, maybe bisect is better, looking at roots. Naming. Eek.

"In geometry, bisection is the division of something into two equal or congruent parts.” (WP) Back to bifurcate. 🙂

I’m trying to convert a Rubyist colleague to Clojure … I’ve tried the obvious points but I may need to admit defeat ... some people just want resources

Different strokes for different folks, but the best part about clojure v. ruby for me: in clojure, I can look at a fn call and know, by and large, exactly what is going to happen. In ruby, with all the mixins and patching monkeys? You really can’t know; you have to actually call it and see what happens (and hope all the effects are visible).

thanks @donaldball … I tried a similar tact … having built a concurrent program from ground up (not just use a web framework) before helps too

at my last gig, we built a distributed file system cache in Java ...

… so … yeah … been there done that

@tankthinks What do you mean by resources?

http://api.rubyonrails.org/classes/ActionDispatch/Routing/Mapper/Resources.html

thinking that's orthogonal

Me too.

@tankthinks have you looked at things like yada?

There are libs that provide such an abstraction for web dev. Consider Liberator.

https://juxt.pro/yada/manual/index.html#_resource_models

oh never looked into yada

i had not looked into liberator or yada

I was simply using compojure and compojure-api

I think you can get a lot of the convenience of Rails’ resources with better HTTP compliance and all defined with data rather than a DSL.

but you’re probably right @camdez and @ddeaguiar … i think these are surface excuses for an underlying “don’t get the whole functional programming paradigm"

I just didn’ have quick answers to … yeah we’ve got an analog to your magic framework

We’ve got comprehensible magic. 🙂

Not sure how close some of the existing web dev libs come to rails but in my experience, explicitness is preferred over magic

that was my point exactly … i come from a deep java background … with too much spring magic

i don’t want magic … i want to know what the f is going on

fwiw, Luke Vanderhart is working on Arachne. It's going to be a framework for building apps with initial focus on web apps.

There's a channel for it if interested #arachne

yup … i’ve read the #arachne blog p

But the general clojure dev experience is composing libs to build what you want

hi - how do i feed the results of one pass of map to the next if mapping over a vector of vectors?

(map #(map (comp f1 f2) %) vec-of-vecs) maybe?

hmmm. turned out my code worked already - now i can't figure out why...

that feels like it’s begging for reductions if you want the intermediate steps and reduce if you don’t.

i am a bear of little brain...

Yeah, reduce sounds right here but I’m not totally sure what the goal is...

It's a little spoiler-y but you might enjoy the conversation in #adventofcode

@mnespor: now I get what we’re doing here. 🙂 http://adventofcode.com/2016/day/2

@oli: regarding your comment, there’s no overlap between the processing of each digit in the final code. Running your code, I get 1945, not 1985, which I believe is expected.

Yep. Verified. Let me know how much of a hint you’re looking for. haha.

you're right. the (extremely long) inputs in the 'real' problem set just happen to eval to the right answer... 😛

Exactly.

69642

which brings me back to how to i pass the result of processing one list in my list-of-lists to the next list to be processed...

@oli: I’ll say this, to avoid spoiling it: map won’t allow you to retain any information from the previous digit, which is essential. You could use reduce, and reach back into the accumulated list of digits every time (I just implemented this). But the cleanest solution is likely going to be to use loop / recur.

One of your bindings in loop is going to be the previously calculated digit (or position).

that's enough to go on, thanks!

@oli: you’re very welcome. Best of luck!

@oli: another small hint: I don’t think you’re gonna end up using that two-arity version of process-instructions.