Fork me on GitHub
#clojure
<
2021-08-26
>
West07:08:59

So I’m trying to programmatically log into a site, but unfortunately I get status 403, reason forbidden. Not sure how to begin to troubleshoot it. I’ve been looking at the browser devtools to try to isolate the missing variable. I tried saving cookies and changing my user agent to a known browser, neither seems to work.

(ns core
  (:require [clj-http.client :as http]))

(let [cs (clj-http.cookies/cookie-store)]
  (http/post "***************************************************"
             {:cookie-store cs
              :headers {"User-Agent" "Mozilla/5.0 (Windows NT 6.1;) Gecko/20100101 Firefox/13.0.1"}
              :form-params {:crsfKey "b081f2098114e7fac1ed87d8a261f577"
:auth "***************************"
                            :password "**************************************"
                            :remember_me "1"
                            :_processLogin "usernamepassword"}}))
Edit: tried adding a csrf-key

thheller07:08:22

@c.westrom commonly sites expects some kind of extra CSRF token for security purposes. you know like ensuring you are logging in from the actual login page not some background automated script 😉

West07:08:04

Hmmmm, any way I can forge that?

thheller07:08:24

if its done correctly no 😛

thheller07:08:49

I mean sure you can always just get the login page and get it out of there but "forging" it no

West07:08:09

Hmmm, ok. Imma try to figure out how to do that.

West07:08:31

I just don’t want to have to use eatoin again.

thheller07:08:29

yeah depends on where they put the data. might just require some html parsing if added as a meta or hidden input. if they do it via JS it might get harder to get

West07:08:06

Aha! I found the token!

West07:08:12

It is in HTML.

West07:08:26

At least for the site I’m logging in to.

West08:08:57

(ns core
  (:require [clj-http.client :as http]
            [clj-http.cookies :as cookies]
            [hickory.core :as hic]
            [hickory.select :as s]))

(defn get-csrf-key
  [body]
  (-> (s/select (s/attr :name #(= % "csrfKey"))
              (-> body
                  hic/parse
                  hic/as-hickory))
    first
    :attrs
    :value))

(let [cs (cookies/cookie-store)
      csrf-key (get-csrf-key (:body (http/get "*****************" {:cookie-store cs})))]
  (http/post "*****************"
             {:cookie-store cs
              :form-params {:csrfKey csrf-key
                            :auth "*********"
                            :password "***********"
                            :remember_me "1"
                            :_processLogin "usernamepassword"}}))
Woohooo!!! I’m gettin it done!

sova-soars-the-sora12:08:31

A great quote from The Joy of Clojure

By relieving the brain of all unnecessary work, a good notation sets it free to
concentrate on more advanced problems.
-- Alfred North Whitehead

✌️ 3
Santiago13:08:57

Given a vector ["foo" "bar" "meh" "baz" "goo"] how would you get the adjacent elements of any arbitrary element? example

(get-adjacent ["foo" "bar" "meh" "baz" "goo"] "baz") ;=> ("meh" "goo")
(get-adjacent ["foo" "bar" "meh" "baz" "goo"] "foo") ;=> (nil "bar") or ("goo" "bar")
I’m thinking of using indices with
(.indexOf v "baz")
and then getting the other indices. Is there a better/more idiomatic way of doing this? effectively generalising into a window centered on an element with radius n

p-himik13:08:01

I'd go with the same approach. The window approach will be both slower and less obvious in its implementation.

3
R.A. Porter13:08:22

Agreed. You could try to do something like this, but it's awfully complex just to handle the degenerate case of a radius of 1 (the radius input is ignored) and returns the wrong answers for both the last element of the vector and an element that is not in the vector. Those can be resolved, but that's even more complexity.

(defn window [m ctr r]
    (reduce (fn [a v]
              (cond
                (= v ctr) (vector a v)
                (vector? a) (reduced (vector (first a) v))
                :default v))
            nil
            m))

Santiago15:08:41

cool, I went with

(defn adjacent-elements
  [v e]
  (let [i (.indexOf v e)]
    (if (< i 0)
      [nil nil]
      [(get v (dec i)) (get v (inc i))])))

donavan15:08:02

Eyeballing that, you probably want to check the end of vector case too

p-himik15:08:22

get will return nil if the index is out of bounds.

👍 3
Ed18:08:37

I've a feeling that Christophe Grand's xforms library has some windowing functions.

Ed19:08:53

Maybe that's useful for a generalized window with a radius of n?

plexus13:08:15

seems there is a discrepancy when using data_readers.clj(c) and read-string I have a tagged reader function which returns a quoted form, which if I'm not mistaken is the right way to go about making cross platform reader functions

;; data_readers.cljc
{ordered/map flatland.ordered.map/ordered-map-reader}

;; ordered/map.clj
(defn ordered-map-reader [args]
  `(ordered-map ~(vec args)))
this works indeed in Clojure and ClojureScript, but when doing a (read-string "#ordered/map []") I get the quoted form back unevaluated: (flatland.ordered.map/ordered-map [])

dominicm13:08:46

This only works in cljs, there's a JIRA about this for cljs to support #? so you can do different implementations as required.

plexus13:08:38

I have seen that ticket, but that seems to be a separate issue. I've done cross-platform readers without that multiple times, just never noticed the read-string behavior

plexus13:08:16

to be clear, a literal #ordered/map [] in the source code works this way in clj and cljs just fine

dominicm13:08:41

To be clear, Clojure doesn't really support returning a quoted for from a tagged reader in order to return an object.

plexus13:08:47

and yet it's behavior I rely on daily

plexus13:08:27

Clojure does evaluate the quoted form when you return it from a reader literal

dominicm13:08:51

Right, but that's not the intended usage afaik.

plexus13:08:43

Interesting, this is news to me.

plexus13:08:26

worked around it as such, which does break in some edge cases, but all in all works fairly well

(defn ordered-map-reader [coll]
  (if (some-> (resolve 'cljs.env/*compiler*) deref)
    `(ordered-map ~(vec coll))
    (ordered-map coll)))

Alex Miller (Clojure team)14:08:01

data reader functions should return data, not code

dominicm14:08:42

objects are a kind of data, right? 😛 (e.g. uuids)

Alex Miller (Clojure team)14:08:14

you are reading a value - return the value

plexus14:08:26

so cross platform reader conditionals are not officially supported?

dominicm14:08:41

No. ClojureScript has different considerations due to serialization.

dominicm14:08:56

I belive you can support cljs in this way by extending a multi-method in cljs.analyzer somewhere

plexus14:08:00

people are doing it in practice though, see for instance https://github.com/henryw374/time-literals

dominicm14:08:59

Right, but in an effort to make that actually work Henry is pursuing the aforementioned cljs jira ticket.

Alex Miller (Clojure team)14:08:12

I can't speak to the cljs concerns, I'm just telling you the Clojure expectations. I'm not sure anyone has actually intentionally considered the design constraints around portable data readers.

Alex Miller (Clojure team)14:08:40

in general, reading does not imply eval (particularly with edn, which is only about reading). the idea is that during read, you can pluggably read a data value that has a tagged literal representation

Alex Miller (Clojure team)14:08:03

data readers in clojure are not macros, and don't have eval (even though you may eval the resulting read value later and this may thus incidentally result in some cases in a value after eval)

3
plexus14:08:39

thanks for clarifying, Alex. That makes sense.

didibus16:08:13

I feel the conceptual model breaks down. I don't think it's that simple. You can execute code in a data_reader, like if you extend edn to parse tagged literal, it's your responsibility now to be sure it can't execute insecure code. And you're expected to execute code as well, since you need to return a in-memory structure from the read form.

didibus16:08:14

I think the issue is just that what is available from the environment at that time is simply undefined it seems.

Alex Miller (Clojure team)16:08:06

the job of a data reader is to take a tagged literal and return a value

Alex Miller (Clojure team)16:08:53

I don't know what is ambiguous about the "environment" and if you're doing anything that requires anything of the environment beyond the tagged form, you're probably doing something you shouldn't.

didibus17:08:57

Well, I mean that taking a form and returning a value from it does involve code execution. What isn't well defined is what that code can expect to depend on. So like in the example, can it depend on the current Clojure runtime? And switch on if Clojure or ClojureScript? Or is this considered out of reach for the code creating the value?

Alex Miller (Clojure team)17:08:03

well you're evaluating the function in the runtime so I would think you could depend on it

Alex Miller (Clojure team)17:08:22

how could it not depend on the runtime

Alex Miller (Clojure team)17:08:33

maybe I just don't understand what you're asking

didibus18:08:39

I think my question is, is there supposed to be a convention that dictates that a data_reader function should be a pure deterministic function? Or is it fine to say create a #time :now that would read from the system clock, or a #slurp "./file"that would read from the file. Etc. And similarly say something that would check a global and return a different value based on it, like checking if the compiler is ClojureScript, if we're running under prod or devo, etc.

Alex Miller (Clojure team)18:08:58

I think most things you mention sound like bad ideas for tagged literals :)

Alex Miller (Clojure team)18:08:34

but Clojure assumes you can make your own decisions :)

didibus18:08:15

Haha, ya that last part is what I was wondering.

henryw37410:08:20

cool, it's not just me who would like data_readers.cljc documented then 🙂 https://github.com/clojure/clojurescript-site/issues/371

Evan Bowling20:08:27

Is there any way to make an extend-protocol call thead local (or am I trying to do something truly ridiculous)?

potetm20:08:47

Maybe an example?

potetm20:08:58

Or more description of what you’re trying to do?

potetm20:08:21

ooooh you want to dispatch to a threadlocal-defined fn?

hiredman20:08:38

No you can't, yes that is ridiculous

potetm20:08:44

what the… you want every thread to behave differently?

potetm20:08:49

yeah if that’s what you want, just define a different fn for every thread to run

ghadi20:08:11

..... walk back -- why are you wanting to do this in the first place?

ghadi20:08:33

testing concern?

Evan Bowling20:08:53

I’ve got some legacy code that extends the clojure.java.jdbc/IResultSetReadColumn in a way to handle ora timestamps but it doesn’t handle timezones gracefully (assumes DB timestamp is in same tz as localhost)

Evan Bowling20:08:17

this logic is used in a number of places so it’s not easy to refactor safely at the moment. That’s ok, thanks for comments all!

potetm21:08:30

Your description is a little vague, but if you need a shim via threadlocal, you could set a dynavar *use-special-timestamp-convert* via with-bindings and have your IResultSetReadColumn implementation check that var.

potetm21:08:51

Cannot recommend that for normal usage, but it might get you out of a jam.

seancorfield22:08:36

@U9BTYPZCZ You could solve that by using next.jdbc for some queries, since it has a more adaptable API. You can create a adapter for a result set builder that can choose to process column values differently.

seancorfield22:08:15

At work, we have both clojure.java.jdbc and next.jdbc in the same codebase. You have to be careful about transactions (see the migration guide) but it's eminently workable.

seancorfield22:08:44

(and in next.jdbc the ReadableColumn protocol is extensible via metadata so a builder adapter's column reader can wrap/annotate column values before read-column-by-index is called)