Json – What's the easiest way to serialize an opaque type with circe
Example: opaque type UserName = String This version is serialized automatically: case class UserName(value: String) extends AnyVal
Ubuntu – InterpreterException in Apache Zeppelin V 0.11.1 executing a paragraph
Problem Description: We tried to start Apache Zeppelin 0.11.1 on the following server. We followed the installation instructions on the official Apache Zepplin website. Apache Zeppelin can be started and the notebooks open. When executing a paragraph, the attached error…
Ubuntu – How to add external jar files in build.sbt in intellij idea?
I was trying to use openCV in Intellij Idea (scala), for that I've downloaded openCV from their official website - and after installing openCV I got opencv-480.jar file in build/bin. For installation I have done (in Ubuntu): $ sudo apt…
Azure – Alternative to InMemoryFileIndex to list files in folder using spark scala
The task I would like to solve: I have a constant influx of files in a specific folder held on azure storage. I would like to periodically list the files in this folder in order to copy them to a…
Json – Unmarshalling protobufs in Scala
I have previously got Unmarshalling working in Scala using ScalaPB and the following marshallers: implicit def marshaller[T <: GeneratedMessage]: ToEntityMarshaller[T] = PredefinedToEntityMarshallers.ByteArrayMarshaller.compose[T](r => r.toByteArray) implicit def unmarshaller[T <: GeneratedMessage with GeneratedMessageCompanion[Request]](implicit companion: GeneratedMessageCompanion[Request]): FromEntityUnmarshaller[Request] = { Unmarshaller.byteArrayUnmarshaller.map[Request](bytes => companion.parseFrom(bytes)) }…
Sending Html Email
I wish to send a HTML email ideally with images I am sending via googles smtp servers, I generate some html with inline css it looks like this <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html charset=UTF-8" /> </head> <body>…
How to install apt-get dependencies in Docker image using sbt-native-packager plugin
I am using the sbt-native-packager plugin to generate a Docker image for a Scala project. The project requires a specific package libnetcdf.so to be installed on the image. I am using the below configuration on my build.sbt dockerCommands ++= Seq(…
Can Scala read JSON with string values without quotes?
I have a log in below format(json/ion) where the value of time_of_initial_request is in timestamp format but without double quotes. {post_handler_type:"trace", default_marketplace:"prod_iad", request_id:"5MTB3656X0JRL38R6LJY", rest_uri:"productApi/feature", client_logging_id:"PredictionMetadata", time_of_initial_request:2023-05-24T15:00:00.577Z } Since the field(key) name does not have quotes, I am using the allowUnquotedFieldNames…
Scala play json: how to capture the whole JSON source?
I receive, in my client, a WSResponse, and use play's deserializeJson method to extract the data, specified by paths, e.g. implicit val lsmf: Format[MyData] = ( (__).formatNullable[JsValue] ~ (__ "id").format[Int] ~ (__ "name").format[String]) (MyData.apply, unlift(MyData.unapply)) The receiving class will look…
Parsing Json with Pyspark Transformations
I have a JSON file of the form: { 42 : {"name": "MeowBark", "id": 42, "category": "pet store"}, 67 : {"name": "Chef's Kiss", "id": 67, "category": "restaurant"}, } I have to parse this in Pyspark and am using the following…