Apache NiFi: Custom Web Scraper Processor – Powered by Selenium

In this article we explore how to build a custom Apache Nifi processor. Our objective is to build a custom NiFi processor, written in Java, that uses Selenium to scrape an arbitrary piece of information off a web-page. The end result will look like this: https://www.youtube.com/watch?v=alRC8owgjl4&feature=youtu.be This highlights the flexibility of Apache NiFi, showing off … Continue reading Apache NiFi: Custom Web Scraper Processor – Powered by Selenium

Advertisements

Lessons learnt: Of Spring Boot + OAuth2 + redirect URIs

TL;DR: make sure NGINX is setup correctly (proxy_set_header) before messing around with your code. Scenario: Deploying a Spring Boot micro-service behind an NGINX reverse proxy gave us issues when using default Google OAuth2 configuration as described here , basically showing the "Redirect URI Mismatch" mentioned at the very end of the linked article Trying the solution based … Continue reading Lessons learnt: Of Spring Boot + OAuth2 + redirect URIs

Lessons learned: Spring Data Postgres application configuration

Scenario: During development of a Spring Boot application, with a PostgreSQL backend, we randomly observe errors such as: hikaripool-1 - connection is not available, request timed out after 30000ms   FATAL: remaining connection slots are reserved for non-replication superuser connections   Solution: There are a few checks to perform: Ensure that you use the correct … Continue reading Lessons learned: Spring Data Postgres application configuration

Reactive Spring: Combining Server-Side Events with Redis PubSub for real-time push events

A simple example of using Redis PubSub and Spring Reactive Server Side Events for real-time push events to the browser Spring Boot v2.0.3.RELEASE Coding using reactive functional style (as opposed to annotation-based) Spring Initializr Dependencies: “Reactive Web” Redis Driver Used: Jedis Language: Kotlin Scenario and design choices The scenario described in this article is depicted below: … Continue reading Reactive Spring: Combining Server-Side Events with Redis PubSub for real-time push events

Reactive Spring: Webflux Multipart File Upload

A clear, simple example of multipart file upload using Reactive Spring Spring Boot v2.0.3.RELEASE Coding using reactive functional style (as opposed to annotation-based) Spring Initializr Dependencies: "Reactive Web" Language: Kotlin Notes follow after the code: https://gist.github.com/dvas0004/fdb63086cd77869066e83a1ca25757d9 Notes: Spring Webflux makes it extremely easy to build reactive web services - but it takes a while to … Continue reading Reactive Spring: Webflux Multipart File Upload

Drill Down into Spring Boot Actuator metrics

We've only seen this very useful feature documented in the official Spring Actuator API Documentation, so maybe not many are aware that you actually have some control over what the Spring Actuator metrics return to your requesting client. The Spring Actuator API allows you to expose several useful metrics that you can use to monitor … Continue reading Drill Down into Spring Boot Actuator metrics

Elasticsearch REST API: JEST upsert

I've already written about tips and tricks when using the Elasticsearch Java API. The Elasticsearch REST API has been going from strength to strength, and it seems that going forward the Elasticsearch team will focus more on the REST API than the native JAVA client. At the time of writing however, the official java REST library … Continue reading Elasticsearch REST API: JEST upsert