RemapVK v0.1

RemapVK is a GPU-based pixel remapper using the Vulkan API.

Parameters

Parameter	Description	Default value
clip	The input clip to be remapped
string "glsl"	Coordinate remapping function written in GLSL	vec2 remap(vec2 f_coords) { return f_coords; }
int "width"	Output video width	Input width
int "height"	Output video height	Input width
int "resampler"	Resampling method	REMAPVK_BICUBIC
int "boundary"	Boundary condition	REMAPVK_BLACK
bool "static"	Static video input	false
bool "gamma"	Approximate gamma correction on appropriate planes	true
bool "multiply"	Enable calculation of a pixel value multiplier for shadows/hightlights	false
string "gpu"	Select GPU by partial name match
bool "debug"	Superimpose GPU list and enable Vulkan validation layers	false

clip

RGB24 and RGB48 clips are converted from and to RGB32 and RGB64 respectively due to Vulkan limitations. YUY2 clips are converted to YV16.

glsl (string)

RemapVK requires a user-specified GLSL function, named remap, taking a vec2 (essentially a float x, y; struct) as input (the x/y coordinates of the destination pixel) and returning another vec2 as output (the x/y coordinates of the source pixel). The following variables are pre-defined:

Variable	Description
vec2 dst_size	Width (dst_size.x) and height (dst_size.y) of output video
vec2 src_size	Width (src_size.x) and height (src_size.y) of input video
int frame_no	Current frame number
int frame_count	Total number of frames
float fps	Video framerate (can be used with frame_no to calculate current time in seconds, for example)

Examples:

GLSL	Input	Output
vec2 remap(vec2 coords) { return vec2( coords.x, coords.y + sin(coords.x / 20 + frame_no) * 10 ); }
vec2 remap(vec2 coords) { float theta = frame_no / 10.0; // angle coords -= src_size * 0.5; // move center to origin mat2 rotate = mat2( cos(theta), sin(theta), -sin(theta), cos(theta) ); return coords * rotate + dst_size * 0.5; }
vec2 remap(vec2 coords) { coords -= dst_size * 0.5; // center to 0,0 float d = length(coords) / (dst_size.y / 2); // normalised distance from center float fov = 1.1; // (half-?)fov in radians float new_d = fov * atan(d / fov); // fisheye correction coords = new_d / d; return coords + src_size 0.5; // undo translation to 0,0 }

width, height (int)

These parameters allow the size of the output to be changed. Because RemapVK remaps from destination pixels to source pixels, operations such as moving the center to 0,0 (e.g. for example) should be performed at the start of remapping using dst_size, and undone before returning using src_size.

resampler (int)

This parameter selects the resampling method. It takes as its value one of four constants:

Value	Description	Example
REMAPVK_NEAREST	Nearest neighbour. Fastest, with no interpolation or supersampling.
REMAPVK_BILINEAR	Bilinear (2×2). Can introduce blurring.
REMAPVK_BICUBIC (default)	Bicubic (4×4). Sharper than bilinear and most mathematically accurate, but introduces ringing (not usually noticeable).
REMAPVK_NEAREST4X4	Nearest neighbour with 4×4 supersampling. This gives sharp edges to zoomed-in pixels and also reduces aliasing in zoomed-out areas. It will also antialias edges if you have discontinuities in the remapping, e.g. hard edges between drawn objects.

boundary (int)

This parameter tells RemapVK how to handle source pixel coordinates that fall outside of the source image. It takes as its value one of four constants:

Value	Description	Example
REMAPVK_BLACK (default)	Fills out-of-bounds areas with black.
REMAPVK_CLAMP	Extends the border pixels.
REMAPVK_REPEAT	Repeats the image.
REMAPVK_MIRROR	Repeats the image with mirroring.

static (bool)

Setting this to true causes RemapVK to only read one input frame (which may not be frame 0), and then to re-use that frame for all output frames. This increases processing speed if the input is a still image.

Note that this parameter differs in effect from the parameter of the same name in my earlier filter, xyremap. Remapping calculations are always performed on each frame.

gamma (bool)

By default, an approximation of gamma (un)correction is applied to Y, R, G, and B channels, so they are blended as (approximately) linear values. This is most useful for sharp RGB graphics:


gamma = true (default)	gamma = false

multiply (bool)

Setting multiply = true enables an alternate mode which, as well as source coordinates, allows you to specify a multiplier value for each pixel. This requires changing the GLSL remap function to match the following example (the parameter names can be changed):

vec2 remap(vec2 coords, out float m) { ... m = ...; // set m to the multiplier value ... }

Setting the value of m returns it to the calling function (in this sense it works like passing by reference in C++). Values of m less than or equal to 1 will multiply the pixel value by m, darkening it. Values of m greater than 1 will brighten the pixel like Photoshop's Screen mode, with values near 1 having no effect and a value of 2 turning the pixel fully white. This can be used to realise light and shade effects, for example:

Script	Input	Output
RemapVK(version, " vec2 remap(vec2 coords, out float m) { m = coords.x / dst_size.x; return coords; } ", multiply = true)
RemapVK(ColorBars(width = 160, height = 120), " vec2 remap(vec2 coords, out float m) { coords -= dst_size * 0.5; float d = length(coords) / length(dst_size * 0.5); m = 1 - d * d; return coords += dst_size * 0.5; } ", multiply = true)
version.assumefps(60) RemapVK(" vec2 remap(vec2 f_coords, out float m) { f_coords -= dst_size / 2; // 0,0 to center float angle = atan(f_coords.y, f_coords.x); float l = length(f_coords); float x = (angle / 3.14159265359 + 1) * src_size.x * 1.5 + frame_no * 2.5; float y = 192000 / l + frame_no * 5 - 52; m = pow(l / length(dst_size) * 2, 0.75); return vec2(x, y); } ", width = 1920, height = 1080,\ multiply = true, static = true,\ boundary = REMAPVK_REPEAT,\ resampler = REMAPVK_NEAREST4X4)
# https://eoimages.gsfc.nasa.gov/images/imagerecords/57000/57735/land_ocean_ice_cloud_2048.jpg imagesource("land_ocean_ice_cloud_2048.jpg", end = 599, fps = 60) RemapVK(" #define RADIUS 450 #define M_PI 3.1415926535897932384626433832795 #define LIGHT (vec3(-1, -1, 1) / sqrt(3)) vec2 remap(vec2 coords, out float m) { coords -= dst_size * 0.5; float d = length(coords); if (d < RADIUS) { // sphere d /= RADIUS; coords /= RADIUS; float z = sqrt(1 - d * d); vec4 point = vec4(coords.x, coords.y, sqrt(1 - d * d), 1); float lat = -acos(point.y) / M_PI - 0.5; float lon = atan(z, point.x) / M_PI; lon += (float(frame_no) / frame_count) * 2; // ambient lighting m = 0.5; // diffuse lighting (cheating by allowing negatives; it looks nicer m += dot(point.xyz, LIGHT) * (1 - m); // specular lighting vec3 rd = reflect(-normalize(LIGHT), point.xyz); // specular m += pow(max(0.0, dot(rd, vec3(0, 0, 1))), 5) * 1.1; return vec2(-lon * src_size.x / 2, lat * src_size.y) + src_size * 0.5; } else { // black m = 0; return vec2(0, 0); // output coordinates don't matter } } ", width = 1000, height = 1000,\ static = true, multiply = true,\ resampler = REMAPVK_NEAREST4X4,\ boundary = REMAPVK_REPEAT)

gpu (string)

Specifying this parameter will cause RemapVK to select the first GPU whose system name contains the parameter value as a substring (case insensitive). E.g., gpu = "nvidia" will select the first NVIDIA GPU.

If unspecified, RemapVK will choose the first dedicated GPU it finds, falling back to the first integrated GPU if there are no dedicated GPUs.

debug (bool)

This parameter enables Vulkan validation layers, which write to debug output, and overlays a list of available GPUs on the video, which is darkened to improve readability:

Note that GPUs may appear in a different order depending on which GPU the hosting program has been assigned to.