This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-03-13
Channels
- # announcements (4)
- # babashka (1)
- # beginners (124)
- # calva (5)
- # cider (3)
- # clara (3)
- # clerk (5)
- # clj-commons (14)
- # cljdoc (12)
- # cljs-dev (14)
- # clojure (43)
- # clojure-austin (23)
- # clojure-europe (55)
- # clojure-nl (1)
- # clojure-norway (11)
- # clojure-uk (2)
- # clojurescript (34)
- # conjure (1)
- # cursive (1)
- # data-science (28)
- # datomic (3)
- # fulcro (20)
- # gratitude (2)
- # hyperfiddle (6)
- # introduce-yourself (1)
- # jobs (5)
- # lsp (56)
- # malli (5)
- # membrane (7)
- # mount (5)
- # off-topic (16)
- # polylith (39)
- # portal (38)
- # practicalli (1)
- # rdf (1)
- # re-frame (8)
- # releases (8)
- # remote-jobs (4)
- # shadow-cljs (49)
- # sql (1)
- # xtdb (36)
I am trying to build some functions to upload and download images to mongodb using monger and grid-fs, and I am using reitit and swagger-ui for the routes. I am able to upload the images to the mongo/gridfs database, that works fine. I am running into problems when trying to download the images in swagger. In a gridfs namespace I have the following functions:
(ns guestbook.db.grid-fs
(:require
[monger.core :as mg :refer
[connect connect-via-uri disconnect get-db get-gridfs]]
[monger.gridfs :as mgfs :refer
[store-file make-input-file filename content-type metadata]]
[monger.collection :as collection]
[monger.conversion :as conv]
[basevan.config :refer [env]]
[ :as io]
[ring.util.http-response :as response])
(:import
[java.nio.file Paths]))
(defn connect-grid-fs []
(connect {:uri (env :database-url)}))
(defn upload [file body]
(let [conn (connect-grid-fs)
db (get-db conn "base")
fs (get-gridfs conn "base")
{:keys [tempfile size filename content-type]} file]
(store-file (make-input-file fs body)
(mgfs/filename filename)
(mgfs/metadata {:format (get-extension filename)})
(mgfs/content-type content-type))))
(defn download [file]
(let [conn (connect-grid-fs)
db (get-db conn "base")
fs (get-gridfs conn "base")]
(mgfs/find-by-filename fs file)))
And for the routes I have imported ring, reitit, swagger, muuntaja etc, now the upload works fine but the download does not, the relevant snippets for the routes are as follows:
(ns guestbook.routes.services
(:require
[guestbook.auth :as auth]
[guestbook.db.core :as db]
[guestbook.db.grid-fs :as grid-fs]
[guestbook.listings.handlers :as listings]
[guestbook.auth.ring :refer
[wrap-authorized get-roles-from-match]]
[guestbook.media :as media]
[guestbook.middleware :as middleware]
[guestbook.middleware.formats :as formats]
[guestbook.responses :as responses]
[ :as io]
[clojure.tools.logging :as log]
[clojure.spec.alpha :as s]
[clojure.string :as string]
[reitit.swagger :as swagger]
[reitit.swagger-ui :as swagger-ui]
[reitit.ring.coercion :as coercion]
[reitit.coercion.spec :as spec-coercion]
[reitit.ring.middleware.muuntaja :as muuntaja]
[reitit.ring.middleware.exception :as exception]
[reitit.ring.middleware.multipart :as multipart]
[reitit.ring.middleware.parameters :as parameters]
[spec-tools.data-spec :as ds]
[ring.middleware.cors :refer [wrap-cors]]
[ring.util.response :as rr]
[ring.util.http-response :as response]
[ring.middleware.multipart-params :as multipart-params]))
;; At the top of the routes I have this:
{:middleware [;; query-params & form-params
parameters/parameters-middleware
;; content-negotiation
muuntaja/format-negotiate-middleware
;; encoding response body
muuntaja/format-response-middleware
;; exception handling
(exception/create-exception-middleware
(assoc exception/default-handlers ::exception/wrap
(fn [handler e request]
(.printStackTrace e)
(handler e request))))
;; cors handling TODO double check this
[wrap-cors
:access-control-allow-origin [#"" #""] ; <-- order matters
:access-control-allow-methods [:get :post :put :patch :delete]
:Access-Control-Allow-Credentials "true"]
exception/exception-middleware
;; decoding request body
muuntaja/format-request-middleware
;; coercing response bodys
coercion/coerce-response-middleware
;; coercing request parameters
coercion/coerce-request-middleware
;; multipart params
multipart/multipart-middleware
;;Our auth middleware
(fn [handler]
(wrap-authorized
handler
(fn handle-unauthorized [req]
(let [route-roles (get-roles-from-match req)]
(log/debug
"Roles for route: "
(:uri req)
route-roles)
(log/debug "User is unauthorized!"
(-> req
:session
:identity
:roles))
(response/forbidden
{:message
(str "User must have one of the following roles: "
route-roles)})))))]
:muuntaja formats/instance
:coercion spec-coercion/coercion
:swagger {:id ::api}}
;; and the two routes I have for the upload and download are below, and like I said the upload works fine, it is the download that is giving me problems:
["/media-name/:name"
{::auth/roles (auth/roles :media/get)
:get {:parameters
{:path {:name string?}}
:handler (fn [{{{:keys [name]} :path} :parameters}]
(if-let [{:keys [data type]}
(grid-fs/download {:name name})]
(-> (io/input-stream data)
(response/ok)
(response/content-type type))
(response/not-found)))}}]
["/media/upload"
{::auth/roles (auth/roles :media/upload)
;; :conflicting true
:post
{:summary "upload a file"
:parameters {:multipart {:file multipart/temp-file-part}}
:handler (fn [{:keys [params body]}]
(let [file (:file params)]
(grid-fs/upload file body)))}}]
but when I try and retrieve the image in the browser using swagger-ui I get this error in the repl:
--- :response---
{:body {:class "java.lang.IllegalArgumentException",
:type "exception"},
:status 500}
--- :response :reitit.ring.middleware.exception/exception ---
{:body {:class "java.lang.IllegalArgumentException",
:type "exception"},
:status 500}
--- :response :reitit.ring.middleware.muuntaja/format-response ---
{:body {:class "java.lang.IllegalArgumentException",
:type "exception"},
:status 500}
--- :response :reitit.ring.middleware.muuntaja/format-negotiate ---
{:body -{:class "java.lang.IllegalArgumentException",
:type "exception"}
+#<http://java.io.ByteArrayInputStream@b346a63>,
:status 500,
+:headers {"Content-Type" "application/json; charset=utf-8"}}
--- :response :reitit.ring.middleware.parameters/parameters ---
{:body #<http://java.io.ByteArrayInputStream@b346a63>,
:headers {"Content-Type" "application/json; charset=utf-8"},
:status 500}
--- :response---
{:body #<http://java.io.ByteArrayInputStream@b346a63>,
:headers {"Content-Type" "application/json; charset=utf-8"},
:status 500}
2023-03-13 03:49:23,081 [cluster-ClusterId{value='640d9b7b69d2d064fa4b2c43', description='null'}-66.88.197.45:20017] DEBUG org.mongodb.driver.protocol.command - Sending command '{ "ismaster" : 1, "$db" : "admin" }' with request id 25466 to database admin on connection [connectionId{localValue:36, serverValue:1159}] to server 66.88.197.45:20017
And the following error in the response in the browser:
Server response
Code Details
500
Error: Internal Server Error
Response body
Download
{
"type": "exception",
"class": "java.lang.IllegalArgumentException"
}
Response headers
content-length: 65
content-type: application/json; charset=utf-8
date: Mon,13 Mar 2023 03:49:22 GMT
server: http-kit
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
Please advisethe server side strack trace should have more info. But from what i'm seeing i don't think you want to set the response content type if you're streaming the result: it should be application/octet-stream
Note that such long questions should be put in a thread or at least have their long code blocks turned into attachments so that the question itself doesn't span a few screens.
{:body {:class "java.lang.IllegalArgumentException",
:type "exception"},
:status 500}
doesn’t tell too muchI'd like to find out if I have unused functions in a library that has quite a large API, and several other internal projects that using it. They are all leinigen projects, and reside in under different folders/repos. Something like this, where both app1 and app2 use lib1:
A couple of ideas I'm going to try:
• configure LSP to handle the multiple projects
• look at lein yagni
Apparently thought about this 8 months ago too 🙂 https://clojurians.slack.com/archives/CPABC1H61/p1656666639369779
you could use clj-kondo analysis: https://github.com/clj-kondo/clj-kondo/tree/master/analysis#unused-vars
which reminds me: when I was doing the same task I used https://github.com/borkdude/carve in report mode with emacs
carve can do this: just give it the paths to everything that should be in scope and then it figures out the unused vars within that collection of code, it doesn't care about lein, just throw the source code at it
Ah interesting. I found this in @U05254DQM’s awesome pro-files!
;; Carve - search through code for unused vars and remove them
;; clojure -M:search/unused-vars --opts '{:paths ["src" "test"]}'
:search/unused-vars
{:extra-deps {borkdude/carve
{:git/url ""
:sha "f45dc3cb35a8b9c6c11d4681f8c673fa347d54be"}}
:main-opts ["-m" "carve.main"]}
If I require a library via Git dependency in my Clojure project and the GitHub repo moves to a different organization, will the redirect GitHub creates for the repository also work for Clojure tools, or would builds break?
(I’m aware that moving a repo to a different organization has a lot of other pitfalls, just trying to understand the technical ramifications)
Got me interested into how the cli does stuff. It seems like it simply shells out to git: https://github.com/clojure/tools.gitlibs/blob/master/src/main/clojure/clojure/tools/gitlibs/impl.clj So if there's any issues with the git integration you could maybe look there for clues.
the CLI determines uses the lib name (by default) to locate the git lib, but you can specify a :git/url to override that. I don't think any of it would work automatically, but can't say I've tried it. It is also possible to use insteadOf git config - that does work.
if you want to see what the CLI is doing wrt git, you can export GITLIBS_DEBUG=true in the env to see all the commands being run
Is there a way to debug reify
not being able to find a method in an interface that definitely has the method defined?
it involves looking at the javadoc for the interface, making sure the javadoc you are looking at is for the version of the library you are using, and then realizing the interface inherits the method from another non-public interface
After poking around at the javadocs, i realized I forgot that I need the "this" initial arg for clojure.
Hi , I am trying to work with on Hazelcast jet with clojure , I was wondering if there is a way lambda expressions in java can be expressed in clojure?
java lambdas will be adapted by the java compiler to implement some SAM(Single Abstract Method) interface/class, so you look at the docs and reify that interface
Please don't cross-post questions in multiple channels. You already asked this -- and provided more details -- in #C053AK3F9
(def node {:value 1
:children [{:value 2
:children [{:value 3
:children []}]}
{:value 4
:children []}]})
(acc-values node)
; =>
(1 2 3 4)
Hey team, how would you accumulate a list of all the values in this tree?
One idea:
(map
:value
(tree-seq
(fn [x] (contains? x :value))
:children
node))
Just discovered tree-seq to do this, but I wonder if there's something betterI would probably just use tree-seq
, but I'm also a big fan of zippers for more complex tree walking. https://github.com/redplanetlabs/specter/wiki/Using-Specter-Recursively#a-basic-example is also a good option.
I was thinking postwalk or zippers, but tree-seq seems great for this
For a regular structure like this I would just write a bespoke function:
(defn acc-values [m] (list* (:value m) (mapcat acc-values (:children m))))
I've stopped using postwalk for treewalking. The other options are typically a similar amount of code and postwalk is less flexible.
The benefit of tree-seq
over acc-values
is that you don't have to worry about stackoverflows
(defn nested-map
([i]
(nested-map {} i))
([m i]
(if (pos? i)
(recur {:children [m]
:value i}
(dec i))
m)))
(->> (tree-seq :children
:children
(nested-map 10000))
(map :value)
count)
;; 10001
(defn acc-values [m]
(list* (:value m) (mapcat acc-values (:children m))))
(count (acc-values (nested-map 10000)))
;; Execution error (StackOverflowError)
yeah. that's something to keep it mind, it depends if you have deeply nested data, 10k levels deep might not be that common