Fork me on GitHub
#hoplon
<
2016-07-07
>
onetom03:07:51

@juhoteperi: i saw you packaged the pdfjs. we are trying it now, but we just get a forever pending promise from (.getDocument js/PDFJS "/some.pdf") do you have any example somewhere which shows how to use it from clojurescript? my suspicion is that it can't access the worker.inc.js. when i try to (set! (.-disableWorker js/PDFJS) true) i get a 404 for

eric.shao05:07:00

Two question : 1. How do you debug 500 error (using castra) ? 2. Why the castra-hoplon report: java.lang.ClassNotFoundException: clojure.lang.Tuple ?

eric.shao05:07:51

The "lein new castra-hoplon template"

eric.shao05:07:11

question1 : there is only “Server error” (.-message error)

eric.shao05:07:38

question2 solved.[ Change clojure1.7 -> clojure1.8

eric.shao06:07:25

question1 solved. browser console show some information.

eric.shao06:07:32

Sorry to bother.

thedavidmeister13:07:32

tbh, i’m pretty worried that the last commit on the webdriver project is 9 months ago and there’s now compatibility issues popping up

thedavidmeister13:07:59

i’m not sure what alternatives are out there with as nice an API as the taxi one

jumblerg14:07:12

@thedavidmeister: onetom’s selenium dependency bump seemed to do the trick for me, once i upgraded to the latest release of firefox. there was a brief period of time i couldn’t run the tests locally, but everything’s working fine now (and on travis as well).

jumblerg14:07:42

i haven’t had any issues with the clojure webdriver lib yet, but if they do arise, my instinct would be to go directly to the java webdriver api moving forward.

jumblerg14:07:23

generally speaking, i’m suspicious that many of the clojure wrappers do more harm than good, as they (1) create an api separate from the already documented api in the host language that isn’t often documented as well (2) fall out of sync with the wrapped api, and (3) add another layer of code where things can go wrong.

jumblerg14:07:45

i don’t have an opinion on this lib in particular though, haven’t looked into it enough.

jumblerg14:07:56

@thedavidmeister: thanks for all your work on the tests, btw, they’ve proven quite helpful.

micha14:07:12

tests awesome

jumblerg14:07:51

it would be awesome if we could get them running with something like saucelabs moving forward, so we can test against all the browsers instead of just ff.

thedavidmeister14:07:16

@jumblerg: yeah that’s all probably fair criticism

thedavidmeister14:07:26

if ff totally stops working i’ll go looking for alternatives

jumblerg14:07:15

everything seems great to me so far. i haven’t had any issues other than that brief period of time ff and selenium were out of sync, and that resolved itself. the tests have been immensely helpful.

dm314:07:33

the problem with webdriver is that the API is so large you can't really wrap it effectively

dm314:07:06

and wrappers sometimes work very well when the API integration can be made data-driven or some unifying abstraction can be found that doesn't work well in Java but works in Clojure

micha14:07:34

the aws sdk is basically data driven already

micha14:07:39

it's all beans

micha14:07:32

i can't use amazonica without the aws sdk javadocs open in front of me

micha14:07:49

since i refuse to use a java ide

onetom15:07:07

i found amazonica quite a pain because the java docs are a bit hard to grasp for me

onetom15:07:30

and i also have to translate between the java method names and the clojure kebab notation

onetom15:07:18

i would rather work with a pure clojure implementation which is based on the original aws rest interface docs

micha15:07:21

you'd end up with the same thing, no?

micha15:07:45

maps in maps in maps, all with magical keys

onetom15:07:59

except probably more testable and more concise

micha15:07:10

i don't see how it would be different

micha15:07:23

except that the beans actually provide reflection capabilities

micha15:07:30

so your ide can show you things

micha15:07:37

if you use a java ide

micha15:07:06

maybe clojure spec can improve on the bean reflection in some way

alandipert15:07:25

i prefer working directly with aws java sdk personally, making judicious use of .. and their "fluent" bean API, the withFoo methods that return new beans

micha15:07:56

i prefer amazonica now, except i've run into some weird coercion issues

micha15:07:07

but i work around them with some try/catch nonsense

micha15:07:35

also i shell out to the aws cli tool when it gets too hairy

onetom15:07:40

i feel i could be happier without such setter/getter babbling: https://github.com/weavejester/clj-aws-s3/blob/master/src/aws/sdk/s3.clj#L52-L74

micha15:07:42

that has worked really well so far

alandipert15:07:02

@onetom: i recommend against using that, it makes credentials harder

onetom15:07:04

ah, true, u are attacking with that jq replacement tool

micha15:07:35

i shell out to the cli tool and consume the json in clojure usually

micha15:07:39

if i shell out

micha15:07:44

cause it will return json

micha15:07:21

so i wrap in try/catch and just submit a byte buffer if it fails there

onetom15:07:51

@thedavidmeister: as i understand the clj-webdriver doesn't need updates really because everything is factored out to the individual browser drivers, which now can evolve on their own, so im not worried about not seeing new commits. in fact it's a good sign probably.

onetom16:07:35

@alandipert: so what do u recommend instead for credential handling?

onetom16:07:26

credentials should be regularly renewed though if they are coming from the aws metadata service

alandipert16:07:00

iirc for the weavejester s3 thing yuo need to manually supply access key and id strings

onetom16:07:05

so im forgiving this complexity to some extent, but still it feels a bit of an overkill

alandipert16:07:10

it doesn't automatically pick them up from the instance role

alandipert16:07:27

but yeah, whether or not that's bad would depend on your deploy setup

micha16:07:16

then i pass that env to clojure.java.shell or whatever as the environment when i shell out

micha16:07:37

so it gets the creds from the instance role that way

alandipert16:07:50

oh i'm talking about in-jvm

alandipert16:07:10

with raw sdk you can do (AmazonS3Client.) and it will try tons of ways to get creds

alandipert16:07:44

with the weavejester s3 lib you need to sniff them out yourself, then pass them to his clj stuff that does https://github.com/weavejester/clj-aws-s3/blob/master/src/aws/sdk/s3.clj#L77-L78

alandipert16:07:52

(defn upload-ctx
  [{:keys [json meta json] :as ctx} & [client]]
  (let [json-str  (json/generate-string json)
        report-id (get meta "ReportId")
        client    (or client (AmazonS3Client.))]
    (.putObject client
                HOENIR_REPORT_OUTPUT_BUCKET
                report-id
                (java.io.ByteArrayInputStream. (.getBytes json-str))
                (doto (ObjectMetadata.)
                  (.addUserMetadata "loginid"   (str (get-in meta ["AdzerkMeta" "loginId"])))
                  (.addUserMetadata "networkid" (str (get meta "NetworkId")))))))

alandipert16:07:00

representative of what i do usually

alandipert16:07:45

and the flipside

(defn get-ctx
  [report-id & [client]]
  (let [client (or client (AmazonS3Client.))
        obj    (.getObject client
                           HOENIR_REPORT_INPUT_BUCKET
                           (format "%s000.gz" report-id))]
    {:csv  (read-csv (GZIPInputStream. (.getObjectContent obj)))
     :meta (get-meta report-id client)}))