35 Commits

Author SHA1 Message Date
87f1748766 Merge e2b0a6a5f2 into ee7e9ef2a3 2025-06-05 15:42:30 +02:00
ee7e9ef2a3 build: setup docker-compose.dev.yml dev iteration
This is based off of the 5-commit branch at
https://github.com/neobooru/szurubooru/blob/docker-development-setup.

Compared to said branch, we
* Exclude extraneous changes such as
    * Any formatting
    * The use of deprecated/ineffectual top-level `version:` in composer files
* Support controlling $THREADS (modernizing the branch to upstream)
* Integrate into master more cleanly

However, client/docker-start-dev uses a temporary hack -- due to
volume mounting overwriting node_modules at arbitrary points during the
`docker compose build` step, we run `npm i` before any given
`npm run watch`.

To see the effects of this commit in action, run:

    docker compose -f ./docker-compose.dev.yml up
2025-05-23 20:05:15 +02:00
376f687c38 chore: questionable is not a recognized rating 2025-02-11 21:50:27 +01:00
4fd848abf2 doc: use docker compose instead of docker-compose
The minimum version requirements are rough guesses, in practice any decently modern docker installation should work.
2025-02-11 21:25:10 +01:00
61b9f81e39 Fixed the google search option in the post details view 2024-11-17 16:48:24 +01:00
b721865931 server/config: generalize container support
Allow running in Kubernetes, podman, and LXC, besides plain docker-compose,
without having to fake out /.dockerenv in non-Docker environments.
2024-11-10 15:44:39 +01:00
Neo
46e3295003 Upload from clipboard (#414)
client/upload: upload from clipboard

Co-authored-by: Eva <evauwu@riseup.net>
2024-09-29 14:54:53 +02:00
031131506e client/css: fix comment word-break
`break-all` makes it hard to read actual comments.
2024-09-29 13:48:06 +02:00
e2b0a6a5f2 Blocklist: Add test drafts and required elements for future blocklist-related tests 2024-05-05 19:34:47 +02:00
9bb11158a3 Blocklist: Fix already existing tests following modifications for blocklist 2024-05-05 19:34:47 +02:00
82721c0bcb Blocklist: Add backend elements:
- Add default blocklist to user when created
- Tags are created if added to a user blocklist
- Add matching migration to DB to add the user blocklist table
- Various other things
2024-05-05 19:34:47 +02:00
e5f61d2c31 Blocklist: Add frontend elements:
- New field in User profile edition to add/remove tags from their blocklist
- This field works as other tag fields, with auto-completion, and a proper list under the textbox
- User must have the right permissions to edit blocklist (either their own or other users')
2024-05-05 19:34:47 +02:00
f8242f8bea Blocklist: Add configuration elements:
- "default_tag_blocklist": string containing a list of space-separated tags to add to a newly created user blocklist
- "default_tag_blocklist_for_anonymous": boolean telling if the above mentionned default blocklist is applied to anonymous users
- Added permissions to edit own blocklist, or others
2024-05-05 19:34:47 +02:00
Neo
d102578b54 Merge pull request #647 from po5/null-checks
client: add null checks
2024-04-27 21:23:16 +02:00
Neo
6edb25d87b Merge pull request #641 from po5/mobile
Mobile improvements
2024-04-26 22:56:58 +02:00
Neo
93fc15f2a4 Merge pull request #642 from po5/better-links 2024-04-26 22:37:54 +02:00
Neo
4f9d46e1c2 Merge branch 'master' into better-links 2024-04-26 22:16:37 +02:00
Eva
b72e81850d client: add null checks 2024-03-28 13:31:48 +01:00
Eva
c1c695f082 client/css: stack bulk tagging toggles horizontally on mobile 2024-03-21 22:26:49 +01:00
Eva
4b6b231fc8 client/posts: reorder elements in mobile layout
Navigation is always right below the image, and comments are always
at the very bottom, to minimize scrolling for common actions.
2024-03-21 22:26:28 +01:00
Eva
6b0c3cfc7f client/html: allow mobile browsers to zoom in 2024-03-21 22:23:45 +01:00
Eva
4ec8cb3ba2 client/css: constrain thumbnails to parent to prevent overextended links 2024-03-21 22:19:46 +01:00
Eva
8d971234a2 client/views: better pool name fallback 2024-03-21 22:16:05 +01:00
Eva
a16bb198ab client/views: more thorough link fallbacks
Prevents a bunch of errors that can happen when a resource is deleted.
2024-03-21 21:53:11 +01:00
Eva
3f182a66ad client/posts: fix overextended tag link 2024-03-21 21:52:52 +01:00
Eva
b52363e82d client/posts: fix overextended download link 2024-03-21 21:52:49 +01:00
Eva
3bf45e4c0a client/users: fix overextended avatar links 2024-03-21 21:52:39 +01:00
5596f53744 posts page ugly horizontal bar fix
fixes ugly horizontal scrollbar appearing when a post with extremely wide image is present in the posts list
2024-02-29 20:56:27 +01:00
da425afc49 Pin pillow-avif-plugin to compatible version range 2024-02-21 17:47:27 +01:00
d7394d672f Fix Pool Search 2024-02-21 01:27:00 +01:00
190d795426 doc: fix small error in pool API docs 2023-12-05 21:31:23 +01:00
7c92ceaf6a fix overflow on comments, prevents ugly unnecesary horizontal scroll 2023-11-05 12:27:03 +01:00
Neo
9e00f37464 Merge pull request #597 from zakame/use-yt-dlp
server/net: use yt-dlp instead of youtube-dl
2023-11-05 12:22:03 +01:00
59c497e168 doc: update for yt-dlp 2023-08-17 20:58:09 +08:00
c292b96f06 server/net: use yt-dlp instead of youtube-dl
youtube-dl no longer even gets URLs properly, so switch to yt-dlp as a
drop-in replacement for it.
2023-08-17 20:41:50 +08:00
54 changed files with 760 additions and 120 deletions

View File

@ -8,7 +8,7 @@ scrubbing](https://sjp.pwn.pl/sjp/;2527372). It is pronounced as *shoorubooru*.
## Features
- Post content: images (JPG, PNG, GIF, animated GIF), videos (MP4, WEBM), Flash animations
- Ability to retrieve web video content using [youtube-dl](https://github.com/ytdl-org/youtube-dl)
- Ability to retrieve web video content using [yt-dlp](https://github.com/yt-dlp/yt-dlp)
- Post comments
- Post notes / annotations, including arbitrary polygons
- Rich JSON REST API ([see documentation](doc/API.md))

View File

@ -1,4 +1,5 @@
node_modules/*
public/
Dockerfile
.dockerignore
**/.gitignore

View File

@ -1,3 +1,26 @@
FROM --platform=$BUILDPLATFORM node:lts-alpine as development
WORKDIR /opt/app
RUN apk --no-cache add \
dumb-init \
nginx \
git
RUN ln -sf /opt/app/nginx.conf.docker /etc/nginx/nginx.conf
RUN rm -rf /var/www
RUN ln -sf /opt/app/public/ /var/www
COPY package.json package-lock.json ./
RUN npm install
ARG BUILD_INFO="docker-development"
ENV BUILD_INFO=${BUILD_INFO}
ENV BACKEND_HOST="server"
CMD ["/opt/app/docker-start-dev.sh"]
VOLUME ["/data"]
FROM --platform=$BUILDPLATFORM node:lts as builder
WORKDIR /opt/app

View File

@ -315,7 +315,7 @@ function makeOutputDirs() {
}
function watch() {
let wss = new WebSocket.Server({ port: 8080 });
let wss = new WebSocket.Server({ port: environment === "development" ? 8081 : 8080 });
const liveReload = !process.argv.includes('--no-live-reload');
function emitReload() {

View File

@ -127,6 +127,10 @@ $comment-border-color = #DDD
color: mix($main-color, $inactive-link-color-darktheme)
.comment-content
p
word-wrap: normal
word-break: break-word
ul, ol
list-style-position: inside
margin: 1em 0

View File

@ -300,10 +300,10 @@ a .access-key
background-size: 20px 20px
img
opacity: 0
width: auto
width: 100%
height: 100%
video
width: auto
width: 100%
height: 100%
.flexbox-dummy

View File

@ -187,6 +187,9 @@
vertical-align: top
@media (max-width: 1000px)
display: block
&.bulk-edit-tags:not(.opened), &.bulk-edit-safety:not(.opened)
float: left
margin-right: 1em
input
margin-bottom: 0.25em
margin-right: 0.25em

View File

@ -15,38 +15,42 @@
border: 0
outline: 0
nav.buttons
margin-top: 0
display: flex
flex-wrap: wrap
article
flex: 1 0 33%
a
display: inline-block
width: 100%
padding: 0.3em 0
text-align: center
vertical-align: middle
transition: background 0.2s linear, box-shadow 0.2s linear
&:not(.inactive):hover
background: lighten($main-color, 90%)
i
font-size: 140%
>.sidebar>nav.buttons, >.content nav.buttons
margin-top: 0
display: flex
flex-wrap: wrap
article
flex: 1 0 33%
a
display: inline-block
width: 100%
padding: 0.3em 0
text-align: center
@media (max-width: 800px)
margin-top: 2em
vertical-align: middle
transition: background 0.2s linear, box-shadow 0.2s linear
&:not(.inactive):hover
background: lighten($main-color, 90%)
i
font-size: 140%
text-align: center
@media (max-width: 800px)
margin-top: 0.6em
margin-bottom: 0.6em
>.content
width: 100%
.post-container
margin-bottom: 2em
margin-bottom: 0.6em
.post-content
margin: 0
.after-mobile-controls
width: 100%
.darktheme .post-view
>.sidebar
>.sidebar, >.content
nav.buttons
article
a:not(.inactive):hover
@ -56,6 +60,8 @@
@media (max-width: 800px)
.post-view
flex-wrap: wrap
>.after-mobile-controls
order: 3
>.sidebar
order: 2
min-width: 100%
@ -113,7 +119,6 @@
h1
margin-bottom: 0.5em
.thumbnail
background-position: 50% 30%
width: 4em
height: 3em
li

View File

@ -21,10 +21,11 @@
.details
font-size: 90%
line-height: 130%
.image
margin: 0.25em 0.6em 0.25em 0
.thumbnail
width: 3em
height: 3em
margin: 0.25em 0.6em 0 0
.darktheme .user-list
ul li

17
client/docker-start-dev.sh Executable file
View File

@ -0,0 +1,17 @@
#!/usr/bin/dumb-init /bin/sh
# Integrate environment variables
sed -i "s|__BACKEND__|${BACKEND_HOST}|" \
/etc/nginx/nginx.conf
# Start server
nginx &
# Watch source for changes and build app
# FIXME: It's not ergonomic to run `npm i` outside of the build step.
# However, the mounting of different directories into the
# client container's /opt/app causes node_modules to disappear
# (the mounting causes client/Dockerfile's RUN npm install
# to silently clobber).
# Find a way to move `npm i` into client/Dockerfile.
npm i && npm run watch -- --polling

View File

@ -2,10 +2,10 @@
# Integrate environment variables
sed -i "s|__BACKEND__|${BACKEND_HOST}|" \
/etc/nginx/nginx.conf
/etc/nginx/nginx.conf
sed -i "s|__BASEURL__|${BASE_URL:-/}|g" \
/var/www/index.htm \
/var/www/manifest.json
/var/www/index.htm \
/var/www/manifest.json
# Start server
exec nginx

View File

@ -2,7 +2,7 @@
<html>
<head>
<meta charset='utf-8'/>
<meta name='viewport' content='width=device-width, initial-scale=1, maximum-scale=1'/>
<meta name='viewport' content='width=device-width, initial-scale=1.0'>
<meta name='theme-color' content='#24aadd'/>
<meta name='apple-mobile-web-app-capable' content='yes'/>
<meta name='apple-mobile-web-app-status-bar-style' content='black'/>

View File

@ -29,6 +29,7 @@
<span class='vim-nav-hint'>Next post &gt;</span>
</a>
</article>
<% if (ctx.canEditPosts || ctx.canDeletePosts || ctx.canFeaturePosts) { %>
<article class='edit-post'>
<% if (ctx.editMode) { %>
<a href='<%= ctx.getPostUrl(ctx.post.id, ctx.parameters) %>'>
@ -36,16 +37,13 @@
<span class='vim-nav-hint'>Back to view mode</span>
</a>
<% } else { %>
<% if (ctx.canEditPosts || ctx.canDeletePosts || ctx.canFeaturePosts) { %>
<a href='<%= ctx.getPostEditUrl(ctx.post.id, ctx.parameters) %>'>
<% } else { %>
<a class='inactive'>
<% } %>
<i class='fa fa-pencil'></i>
<span class='vim-nav-hint'>Edit post</span>
<a href='<%= ctx.getPostEditUrl(ctx.post.id, ctx.parameters) %>'>
<i class='fa fa-pencil'></i>
<span class='vim-nav-hint'>Edit post</span>
</a>
<% } %>
</article>
<% } %>
</nav>
<div class='sidebar-container'></div>
@ -54,13 +52,15 @@
<div class='content'>
<div class='post-container'></div>
<% if (ctx.canListComments) { %>
<div class='comments-container'></div>
<% } %>
<div class='after-mobile-controls'>
<% if (ctx.canCreateComments) { %>
<h2>Add comment</h2>
<div class='comment-form-container'></div>
<% } %>
<% if (ctx.canCreateComments) { %>
<h2>Add comment</h2>
<div class='comment-form-container'></div>
<% } %>
<% if (ctx.canListComments) { %>
<div class='comments-container'></div>
<% } %>
</div>
</div>
</div>

View File

@ -17,8 +17,8 @@
'video/mp4': 'MPEG-4',
'video/quicktime': 'MOV',
'application/x-shockwave-flash': 'SWF',
}[ctx.post.mimeType] %>
</a>
}[ctx.post.mimeType] %><!--
--></a>
(<%- ctx.post.canvasWidth %>x<%- ctx.post.canvasHeight %>)
<% if (ctx.post.flags.length) { %><!--
--><% if (ctx.post.flags.includes('loop')) { %><i class='fa fa-repeat'></i><% } %><!--
@ -58,7 +58,7 @@
Search on
<a href='http://iqdb.org/?url=<%- encodeURIComponent(ctx.post.fullContentUrl) %>'>IQDB</a> &middot;
<a href='https://danbooru.donmai.us/posts?tags=md5:<%- ctx.post.checksumMD5 %>'>Danbooru</a> &middot;
<a href='https://www.google.com/searchbyimage?&image_url=<%- encodeURIComponent(ctx.post.fullContentUrl) %>'>Google Images</a>
<a href='https://lens.google.com/uploadbyurl?url=<%- encodeURIComponent(ctx.post.fullContentUrl) %>'>Google Images</a>
</section>
<section class='social'>
@ -99,10 +99,10 @@
--><% if (ctx.canListPosts) { %><!--
--><a href='<%- ctx.formatClientLink('posts', {query: ctx.escapeTagName(tag.names[0])}) %>' class='<%= ctx.makeCssName(tag.category, 'tag') %>'><!--
--><% } %><!--
--><%- ctx.getPrettyName(tag.names[0]) %>&#32;<!--
--><%- ctx.getPrettyName(tag.names[0]) %><!--
--><% if (ctx.canListPosts) { %><!--
--></a><!--
--><% } %><!--
--><% } %>&#32;<!--
--><span class='tag-usages' data-pseudo-content='<%- tag.postCount %>'></span><!--
--></li><!--
--><% } %><!--

View File

@ -16,7 +16,6 @@
%><form class='horizontal bulk-edit bulk-edit-tags'><%
%><span class='append hint'>Tagging with:</span><%
%><a href class='mousetrap button append open'>Mass tag</a><%
%><wbr/><%
%><%= ctx.makeTextInput({name: 'tag', value: ctx.parameters.tag}) %><%
%><input class='mousetrap start' type='submit' value='Start tagging'/><%
%><a href class='mousetrap button append close'>Stop tagging</a><%

View File

@ -68,6 +68,12 @@
</div>
</li>
<% } %>
<% if (ctx.canEditBlocklist) { %>
<li class='blocklist'>
<%= ctx.makeTextInput({text: 'Blocklist'}) %>
</li>
<% } %>
</ul>
<div class='messages'></div>

View File

@ -91,16 +91,16 @@ class PoolController {
_evtUpdate(e) {
this._view.clearMessages();
this._view.disableForm();
if (e.detail.names !== undefined) {
if (e.detail.names !== undefined && e.detail.names !== null) {
e.detail.pool.names = e.detail.names;
}
if (e.detail.category !== undefined) {
if (e.detail.category !== undefined && e.detail.category !== null) {
e.detail.pool.category = e.detail.category;
}
if (e.detail.description !== undefined) {
if (e.detail.description !== undefined && e.detail.description !== null) {
e.detail.pool.description = e.detail.description;
}
if (e.detail.posts !== undefined) {
if (e.detail.posts !== undefined && e.detail.posts !== null) {
e.detail.pool.posts.clear();
for (let postId of e.detail.posts) {
e.detail.pool.posts.add(

View File

@ -43,6 +43,8 @@ class PoolListController {
this._headerView.addEventListener(
"submit",
(e) => this._evtSubmit(e),
);
this._headerView.addEventListener(
"navigate",
(e) => this._evtNavigate(e)
);

View File

@ -169,22 +169,22 @@ class PostMainController extends BasePostController {
this._view.sidebarControl.disableForm();
this._view.sidebarControl.clearMessages();
const post = e.detail.post;
if (e.detail.safety !== undefined) {
if (e.detail.safety !== undefined && e.detail.safety !== null) {
post.safety = e.detail.safety;
}
if (e.detail.flags !== undefined) {
if (e.detail.flags !== undefined && e.detail.flags !== null) {
post.flags = e.detail.flags;
}
if (e.detail.relations !== undefined) {
if (e.detail.relations !== undefined && e.detail.relations !== null) {
post.relations = e.detail.relations;
}
if (e.detail.content !== undefined) {
if (e.detail.content !== undefined && e.detail.content !== null) {
post.newContent = e.detail.content;
}
if (e.detail.thumbnail !== undefined) {
if (e.detail.thumbnail !== undefined && e.detail.thumbnail !== null) {
post.newThumbnail = e.detail.thumbnail;
}
if (e.detail.source !== undefined) {
if (e.detail.source !== undefined && e.detail.source !== null) {
post.source = e.detail.source;
}
post.save().then(

View File

@ -95,13 +95,13 @@ class TagController {
_evtUpdate(e) {
this._view.clearMessages();
this._view.disableForm();
if (e.detail.names !== undefined) {
if (e.detail.names !== undefined && e.detail.names !== null) {
e.detail.tag.names = e.detail.names;
}
if (e.detail.category !== undefined) {
if (e.detail.category !== undefined && e.detail.category !== null) {
e.detail.tag.category = e.detail.category;
}
if (e.detail.description !== undefined) {
if (e.detail.description !== undefined && e.detail.description !== null) {
e.detail.tag.description = e.detail.description;
}
e.detail.tag.save().then(

View File

@ -89,6 +89,7 @@ class UserController {
canEditAvatar: api.hasPrivilege(
`users:edit:${infix}:avatar`
),
canEditBlocklist: api.hasPrivilege(`users:edit:${infix}:blocklist`),
canEditAnything: api.hasPrivilege(`users:edit:${infix}`),
canListTokens: api.hasPrivilege(
`userTokens:list:${infix}`
@ -175,21 +176,21 @@ class UserController {
const isLoggedIn = api.isLoggedIn(e.detail.user);
const infix = isLoggedIn ? "self" : "any";
if (e.detail.name !== undefined) {
if (e.detail.name !== undefined && e.detail.name !== null) {
e.detail.user.name = e.detail.name;
}
if (e.detail.email !== undefined) {
if (e.detail.email !== undefined && e.detail.email !== null) {
e.detail.user.email = e.detail.email;
}
if (e.detail.rank !== undefined) {
if (e.detail.rank !== undefined && e.detail.rank !== null) {
e.detail.user.rank = e.detail.rank;
}
if (e.detail.password !== undefined) {
if (e.detail.password !== undefined && e.detail.password !== null) {
e.detail.user.password = e.detail.password;
}
if (e.detail.avatarStyle !== undefined) {
if (e.detail.avatarStyle !== undefined && e.detail.avatarStyle !== null) {
e.detail.user.avatarStyle = e.detail.avatarStyle;
if (e.detail.avatarContent) {
e.detail.user.avatarContent = e.detail.avatarContent;
@ -302,7 +303,7 @@ class UserController {
this._view.clearMessages();
this._view.disableForm();
if (e.detail.note !== undefined) {
if (e.detail.note !== undefined && e.detail.note !== null) {
e.detail.userToken.note = e.detail.note;
}

View File

@ -48,6 +48,12 @@ class FileDropperControl extends events.EventTarget {
this._urlInputNode.addEventListener("keydown", (e) =>
this._evtUrlInputKeyDown(e)
);
this._urlInputNode.addEventListener("paste", (e) => {
// document.onpaste is used on the post-upload page.
// And this event is used on the post edit page.
if (document.getElementById("post-upload")) return;
this._evtPaste(e)
});
}
if (this._urlConfirmButtonNode) {
this._urlConfirmButtonNode.addEventListener("click", (e) =>
@ -55,6 +61,11 @@ class FileDropperControl extends events.EventTarget {
);
}
document.onpaste = (e) => {
if (!document.getElementById("post-upload")) return;
this._evtPaste(e)
}
this._originalHtml = this._dropperNode.innerHTML;
views.replaceContent(target, source);
}
@ -129,6 +140,17 @@ class FileDropperControl extends events.EventTarget {
this._emitFiles(e.dataTransfer.files);
}
_evtPaste(e) {
const items = (e.clipboardData || e.originalEvent.clipboardData).items;
const fileList = Array.from(items).map((x) => x.getAsFile()).filter(f => f);
if (!this._options.allowMultiple && fileList.length > 1) {
window.alert("Cannot select multiple files.");
} else if (fileList.length > 0) {
this._emitFiles(fileList);
}
}
_evtUrlInputKeyDown(e) {
if (e.which !== KEY_RETURN) {
return;

View File

@ -103,6 +103,30 @@ class PostContentControl {
}
_refreshSize() {
if (window.innerWidth <= 800) {
const buttons = document.querySelector(".sidebar > .buttons");
if (buttons) {
const content = document.querySelector(".content");
content.insertBefore(buttons, content.querySelector(".post-container + *"));
const afterControls = document.querySelector(".content > .after-mobile-controls");
if (afterControls) {
afterControls.parentElement.parentElement.appendChild(afterControls);
}
}
} else {
const buttons = document.querySelector(".content > .buttons");
if (buttons) {
const sidebar = document.querySelector(".sidebar");
sidebar.insertBefore(buttons, sidebar.firstElementChild);
}
const afterControls = document.querySelector(".content + .after-mobile-controls");
if (afterControls) {
document.querySelector(".content").appendChild(afterControls);
}
}
this._currentFitFunction();
}

View File

@ -427,7 +427,7 @@ class PostEditSidebarControl extends events.EventTarget {
: undefined,
thumbnail:
this._newPostThumbnail !== undefined
this._newPostThumbnail !== undefined && this._newPostThumbnail !== null
? this._newPostThumbnail
: undefined,

View File

@ -3,7 +3,7 @@
const config = require("./config.js");
if (config.environment == "development") {
var ws = new WebSocket("ws://" + location.hostname + ":8080");
var ws = new WebSocket("ws://" + location.hostname + ":8081");
ws.addEventListener("open", function (event) {
console.log("Live-reloading websocket connected.");
});

View File

@ -271,7 +271,7 @@ class Post extends events.EventTarget {
if (this._newContent) {
files.content = this._newContent;
}
if (this._newThumbnail !== undefined) {
if (this._newThumbnail !== undefined && this._newThumbnail !== null) {
files.thumbnail = this._newThumbnail;
}
if (this._source !== this._orig._source) {

View File

@ -3,11 +3,19 @@
const api = require("../api.js");
const uri = require("../util/uri.js");
const events = require("../events.js");
const misc = require("../util/misc.js");
class User extends events.EventTarget {
constructor() {
const TagList = require("./tag_list.js");
super();
this._orig = {};
for (let obj of [this, this._orig]) {
obj._blocklist = new TagList();
}
this._updateFromResponse({});
}
@ -71,6 +79,10 @@ class User extends events.EventTarget {
throw "Invalid operation";
}
get blocklist() {
return this._blocklist;
}
set name(value) {
this._name = value;
}
@ -95,6 +107,10 @@ class User extends events.EventTarget {
this._password = value;
}
set blocklist(value) {
this._blocklist = value || "";
}
static fromResponse(response) {
const ret = new User();
ret._updateFromResponse(response);
@ -121,6 +137,11 @@ class User extends events.EventTarget {
if (this._rank !== this._orig._rank) {
detail.rank = this._rank;
}
if (misc.arraysDiffer(this._blocklist, this._orig._blocklist)) {
detail.blocklist = this._blocklist.map(
(relation) => relation.names[0]
);
}
if (this._avatarStyle !== this._orig._avatarStyle) {
detail.avatarStyle = this._avatarStyle;
}
@ -187,6 +208,10 @@ class User extends events.EventTarget {
_dislikedPostCount: response.dislikedPostCount,
};
for (let obj of [this, this._orig]) {
obj._blocklist.sync(response.blocklist);
}
Object.assign(this, map);
Object.assign(this._orig, map);

View File

@ -209,13 +209,13 @@ function makePostLink(id, includeHash) {
}
function makeTagLink(name, includeHash, includeCount, tag) {
const category = tag ? tag.category : "unknown";
const category = tag && tag.category ? tag.category : "unknown";
let text = misc.getPrettyName(name);
if (includeHash === true) {
text = "#" + text;
}
if (includeCount === true) {
text += " (" + (tag ? tag.postCount : 0) + ")";
text += " (" + (tag && tag.postCount ? tag.postCount : 0) + ")";
}
return api.hasPrivilege("tags:view")
? makeElement(
@ -234,15 +234,15 @@ function makeTagLink(name, includeHash, includeCount, tag) {
}
function makePoolLink(id, includeHash, includeCount, pool, name) {
const category = pool ? pool.category : "unknown";
const category = pool && pool.category ? pool.category : "unknown";
let text = misc.getPrettyName(
name ? name : pool ? pool.names[0] : "unknown"
name ? name : pool && pool.names ? pool.names[0] : "pool " + id
);
if (includeHash === true) {
text = "#" + text;
}
if (includeCount === true) {
text += " (" + (pool ? pool.postCount : 0) + ")";
text += " (" + (pool && pool.postCount ? pool.postCount : 0) + ")";
}
return api.hasPrivilege("pools:view")
? makeElement(
@ -264,7 +264,7 @@ function makeUserLink(user) {
let text = makeThumbnail(user ? user.avatarUrl : null);
text += user && user.name ? misc.escapeHtml(user.name) : "Anonymous";
const link =
user && api.hasPrivilege("users:view")
user && user.name && api.hasPrivilege("users:view")
? makeElement(
"a",
{ href: uri.formatClientLink("user", user.name) },

View File

@ -4,6 +4,8 @@ const events = require("../events.js");
const api = require("../api.js");
const views = require("../util/views.js");
const FileDropperControl = require("../controls/file_dropper_control.js");
const TagInputControl = require("../controls/tag_input_control.js")
const misc = require("../util/misc.js");
const template = views.getTemplate("user-edit");
@ -41,6 +43,13 @@ class UserEditView extends events.EventTarget {
});
}
if (this._blocklistFieldNode) {
new TagInputControl(
this._blocklistFieldNode,
this._user.blocklist
);
}
this._formNode.addEventListener("submit", (e) => this._evtSubmit(e));
}
@ -83,6 +92,10 @@ class UserEditView extends events.EventTarget {
? this._rankInputNode.value
: undefined,
blocklist: this._blocklistFieldNode
? misc.splitByWhitespace(this._blocklistFieldNode.value)
: undefined,
avatarStyle: this._avatarStyleInputNode
? this._avatarStyleInputNode.value
: undefined,
@ -101,6 +114,10 @@ class UserEditView extends events.EventTarget {
return this._hostNode.querySelector("form");
}
get _blocklistFieldNode() {
return this._formNode.querySelector(".blocklist input");
}
get _rankInputNode() {
return this._formNode.querySelector("[name=rank]");
}

View File

@ -54,7 +54,7 @@
- [Deleting pool category](#deleting-pool-category)
- [Setting default pool category](#setting-default-pool-category)
- Pools
- [Listing pools](#listing-pool)
- [Listing pools](#listing-pools)
- [Creating pool](#creating-pool)
- [Updating pool](#updating-pool)
- [Getting pool](#getting-pool)
@ -165,9 +165,9 @@ way. The files, however, should be passed as regular fields appended with a
accepts a file named `content`, the client should pass
`{"contentUrl":"http://example.com/file.jpg"}` as a part of the JSON message
body. When creating or updating post content using this method, the server can
also be configured to employ [youtube-dl](https://github.com/ytdl-org/youtube-dl)
to download content from popular sites such as youtube, gfycat, etc. Access to
youtube-dl can be configured with the `'uploads:use_downloader'` permission
also be configured to employ [yt-dlp](https://github.com/yt-dlp/yt-dlp) to
download content from popular sites such as youtube, gfycat, etc. Access to
yt-dlp can be configured with the `'uploads:use_downloader'` permission
Finally, in some cases the user might want to reuse one file between the
requests to save the bandwidth (for example, reverse search + consecutive
@ -789,7 +789,7 @@ data.
| `fav-time` | alias of `fav-date` |
| `feature-date` | featured at given date |
| `feature-time` | alias of `feature-time` |
| `safety` | having given safety. `<value>` can be either `safe`, `sketchy` (or `questionable`) or `unsafe`. |
| `safety` | having given safety. `<value>` can be either `safe`, `sketchy` or `unsafe`. |
| `rating` | alias of `safety` |
**Sort style tokens**
@ -1389,7 +1389,7 @@ data.
## Creating pool
- **Request**
`POST /pools/create`
`POST /pool`
- **Input**

View File

@ -1,5 +1,5 @@
This assumes that you have Docker (version 17.05 or greater)
and Docker Compose (version 1.6.0 or greater) already installed.
This assumes that you have Docker (version 19.03 or greater)
and the Docker Compose CLI (version 1.27.0 or greater) already installed.
### Prepare things
@ -38,7 +38,7 @@ and Docker Compose (version 1.6.0 or greater) already installed.
This pulls the latest containers from docker.io:
```console
user@host:szuru$ docker-compose pull
user@host:szuru$ docker compose pull
```
If you have modified the application's source and would like to manually
@ -49,17 +49,17 @@ and Docker Compose (version 1.6.0 or greater) already installed.
For first run, it is recommended to start the database separately:
```console
user@host:szuru$ docker-compose up -d sql
user@host:szuru$ docker compose up -d sql
```
To start all containers:
```console
user@host:szuru$ docker-compose up -d
user@host:szuru$ docker compose up -d
```
To view/monitor the application logs:
```console
user@host:szuru$ docker-compose logs -f
user@host:szuru$ docker compose logs -f
# (CTRL+C to exit)
```
@ -84,13 +84,13 @@ and Docker Compose (version 1.6.0 or greater) already installed.
2. Build the containers:
```console
user@host:szuru$ docker-compose build
user@host:szuru$ docker compose build
```
That will attempt to build both containers, but you can specify `client`
or `server` to make it build only one.
If `docker-compose build` spits out:
If `docker compose build` spits out:
```
ERROR: Service 'server' failed to build: failed to parse platform : "" is an invalid component of "": platform specifier component must match "^[A-Za-z0-9_-]+$": invalid argument
@ -102,7 +102,7 @@ and Docker Compose (version 1.6.0 or greater) already installed.
user@host:szuru$ export DOCKER_BUILDKIT=1; export COMPOSE_DOCKER_CLI_BUILD=1
```
...and run `docker-compose build` again.
...and run `docker compose build` again.
*Note: If your changes are not taking effect in your builds, consider building
with `--no-cache`.*
@ -117,7 +117,7 @@ with `--no-cache`.*
run from docker:
```console
user@host:szuru$ docker-compose run server ./szuru-admin --help
user@host:szuru$ docker compose run server ./szuru-admin --help
```
will give you a breakdown on all available commands.

54
docker-compose.dev.yml Normal file
View File

@ -0,0 +1,54 @@
## Docker Compose configuration for dev iteration
##
## Data is transient by using named vols.
## Run: docker-compose -f ./docker-compose.dev.yml up
services:
server:
build:
context: ./server
target: development
depends_on:
- sql
environment:
## These should be the names of the dependent containers listed below,
## or FQDNs/IP addresses if these services are running outside of Docker
POSTGRES_HOST: sql
## Credentials for database:
POSTGRES_USER:
POSTGRES_PASSWORD:
## Commented Values are Default:
#POSTGRES_DB: defaults to same as POSTGRES_USER
#POSTGRES_PORT: 5432
#LOG_SQL: 0 (1 for verbose SQL logs)
THREADS:
volumes:
- "data:/data"
- "./server/:/opt/app/"
client:
build:
context: ./client
target: development
depends_on:
- server
volumes:
- "data:/data:ro"
- "./client/:/opt/app/"
- "/opt/app/public/"
ports:
- "${PORT}:80"
- "8081:8081"
sql:
image: postgres:11-alpine
restart: unless-stopped
environment:
POSTGRES_USER:
POSTGRES_PASSWORD:
volumes:
- "sql:/var/lib/postgresql/data"
volumes:
data:
sql:

View File

@ -1,9 +1,7 @@
## Example Docker Compose configuration
##
## Use this as a template to set up docker-compose, or as guide to set up other
## Use this as a template to set up docker compose, or as guide to set up other
## orchestration services
version: '2'
services:
server:

View File

@ -23,15 +23,15 @@ RUN apk --no-cache add \
py3-pillow \
py3-pynacl \
py3-tz \
py3-pyrfc3339 \
&& pip3 install --no-cache-dir --disable-pip-version-check \
py3-pyrfc3339
RUN pip3 install --no-cache-dir --disable-pip-version-check \
"alembic>=0.8.5" \
"coloredlogs==5.0" \
"pyheif==0.6.1" \
"heif-image-plugin>=0.3.2" \
youtube_dl \
"pillow-avif-plugin>=1.1.0" \
&& apk --no-cache del py3-pip
yt-dlp \
"pillow-avif-plugin~=1.1.0"
RUN apk --no-cache del py3-pip
COPY ./ /opt/app/
RUN rm -rf /opt/app/szurubooru/tests
@ -61,7 +61,42 @@ ENTRYPOINT ["pytest", "--tb=short"]
CMD ["szurubooru/"]
FROM prereqs as development
WORKDIR /opt/app
ARG PUID=1000
ARG PGID=1000
RUN apk --no-cache add \
dumb-init \
py3-pip \
py3-setuptools \
py3-waitress \
&& pip3 install --no-cache-dir --disable-pip-version-check \
hupper \
&& mkdir -p /opt/app /data \
&& addgroup -g ${PGID} app \
&& adduser -SDH -h /opt/app -g '' -G app -u ${PUID} app \
&& chown -R app:app /opt/app /data
USER app
CMD ["/opt/app/docker-start-dev.sh"]
ARG PORT=6666
ENV PORT=${PORT}
EXPOSE ${PORT}
ARG THREADS=4
ENV THREADS=${THREADS}
VOLUME ["/data/"]
FROM prereqs as release
COPY ./ /opt/app/
RUN rm -rf /opt/app/szurubooru/tests
WORKDIR /opt/app
ARG PUID=1000

View File

@ -67,6 +67,12 @@ webhooks:
default_rank: regular
# default blocklisted tags (space separated)
default_tag_blocklist: ''
# Apply blocklist for anonymous viewers too
default_tag_blocklist_for_anonymous: yes
privileges:
'users:create:self': anonymous # Registration permission
'users:create:any': administrator
@ -76,11 +82,13 @@ privileges:
'users:edit:any:pass': moderator
'users:edit:any:email': moderator
'users:edit:any:avatar': moderator
'users:edit:any:blocklist': moderator
'users:edit:any:rank': moderator
'users:edit:self:name': regular
'users:edit:self:pass': regular
'users:edit:self:email': regular
'users:edit:self:avatar': regular
'users:edit:self:blocklist': regular
'users:edit:self:rank': moderator # one can't promote themselves or anyone to upper rank than their own.
'users:delete:any': administrator
'users:delete:self': regular

8
server/docker-start-dev.sh Executable file
View File

@ -0,0 +1,8 @@
#!/usr/bin/dumb-init /bin/sh
set -e
cd /opt/app
alembic upgrade head
echo "Starting szurubooru API on port ${PORT} - Running on ${THREADS} threads"
exec hupper -m waitress --port ${PORT} --threads ${THREADS} szurubooru.facade:app

View File

@ -3,7 +3,7 @@ certifi>=2017.11.5
coloredlogs==5.0
heif-image-plugin==0.3.2
numpy>=1.8.2
pillow-avif-plugin>=1.1.0
pillow-avif-plugin~=1.1.0
pillow>=4.3.0
psycopg2-binary>=2.6.1
pyheif==0.6.1
@ -12,4 +12,4 @@ pyRFC3339>=1.0
pytz>=2018.3
pyyaml>=3.11
SQLAlchemy>=1.0.12, <1.4
youtube_dl
yt-dlp

View File

@ -43,6 +43,8 @@ def get_info(ctx: rest.Context, _params: Dict[str, str] = {}) -> rest.Response:
"tagNameRegex": config.config["tag_name_regex"],
"tagCategoryNameRegex": config.config["tag_category_name_regex"],
"defaultUserRank": config.config["default_rank"],
"defaultTagBlocklist": config.config["default_tag_blocklist"],
"defaultTagBlocklistForAnonymous": config.config["default_tag_blocklist_for_anonymous"],
"enableSafety": config.config["enable_safety"],
"contactEmail": config.config["contact_email"],
"canSendMails": bool(config.config["smtp"]["host"]),

View File

@ -2,7 +2,7 @@ from datetime import datetime
from typing import Dict, List, Optional
from szurubooru import db, model, rest, search
from szurubooru.func import auth, serialization, snapshots, tags, versions
from szurubooru.func import auth, serialization, snapshots, tags, versions, users
_search_executor = search.Executor(search.configs.TagSearchConfig())

View File

@ -1,7 +1,7 @@
from typing import Any, Dict
from typing import Any, Dict, List
from szurubooru import model, rest, search
from szurubooru.func import auth, serialization, users, versions
from szurubooru import db, model, rest, search
from szurubooru.func import auth, serialization, snapshots, users, versions, tags
_search_executor = search.Executor(search.configs.UserSearchConfig())
@ -17,6 +17,18 @@ def _serialize(
)
def _create_tag_if_needed(tag_names: List[str], user: model.User) -> None:
# Taken from tag_api.py
if not tag_names:
return
_existing_tags, new_tags = tags.get_or_create_tags_by_names(tag_names)
if len(new_tags):
auth.verify_privilege(user, "tags:create")
db.session.flush()
for tag in new_tags:
snapshots.create(tag, user)
@rest.routes.get("/users/?")
def get_users(
ctx: rest.Context, _params: Dict[str, str] = {}
@ -50,6 +62,10 @@ def create_user(
)
ctx.session.add(user)
ctx.session.commit()
to_add, _ = users.update_user_blocklist(user, None)
for e in to_add:
ctx.session.add(e)
ctx.session.commit()
return _serialize(ctx, user, force_show_email=True)
@ -80,6 +96,16 @@ def update_user(ctx: rest.Context, params: Dict[str, str]) -> rest.Response:
if ctx.has_param("rank"):
auth.verify_privilege(ctx.user, "users:edit:%s:rank" % infix)
users.update_user_rank(user, ctx.get_param_as_string("rank"), ctx.user)
if ctx.has_param("blocklist"):
auth.verify_privilege(ctx.user, "users:edit:%s:blocklist" % infix)
blocklist = ctx.get_param_as_string_list("blocklist")
_create_tag_if_needed(blocklist, user) # Non-existing tags are created.
blocklist_tags = tags.get_tags_by_names(blocklist)
to_add, to_remove = users.update_user_blocklist(user, blocklist_tags)
for e in to_remove:
ctx.session.delete(e)
for e in to_add:
ctx.session.add(e)
if ctx.has_param("avatarStyle"):
auth.verify_privilege(ctx.user, "users:edit:%s:avatar" % infix)
users.update_user_avatar(

View File

@ -21,7 +21,7 @@ def _merge(left: Dict, right: Dict) -> Dict:
return left
def _docker_config() -> Dict:
def _container_config() -> Dict:
if "TEST_ENVIRONMENT" not in os.environ:
for key in ["POSTGRES_USER", "POSTGRES_PASSWORD", "POSTGRES_HOST"]:
if key not in os.environ:
@ -49,6 +49,15 @@ def _file_config(filename: str) -> Dict:
return yaml.load(handle.read(), Loader=yaml.SafeLoader) or {}
def _running_inside_container() -> bool:
env = os.environ.keys()
return (
os.path.exists("/.dockerenv")
or "KUBERNETES_SERVICE_HOST" in env
or "container" in env # set by lxc/podman
)
def _read_config() -> Dict:
ret = _file_config("config.yaml.dist")
if os.path.isfile("config.yaml"):
@ -57,8 +66,8 @@ def _read_config() -> Dict:
logger.warning(
"'config.yaml' should be a file, not a directory, skipping"
)
if os.path.exists("/.dockerenv"):
ret = _merge(ret, _docker_config())
if _running_inside_container():
ret = _merge(ret, _container_config())
return ret

View File

@ -64,7 +64,7 @@ def download(url: str, use_video_downloader: bool = False) -> bytes:
def _get_youtube_dl_content_url(url: str) -> str:
cmd = ["youtube-dl", "--format", "best", "--no-playlist"]
cmd = ["yt-dlp", "--format", "best", "--no-playlist"]
if config.config["user_agent"]:
cmd.extend(["--user-agent", config.config["user_agent"]])
cmd.extend(["--get-url", url])

View File

@ -159,6 +159,9 @@ def get_tag_by_name(name: str) -> model.Tag:
def get_tags_by_names(names: List[str]) -> List[model.Tag]:
"""
Returns a list of all tags which names include all the letters from the input list
"""
names = util.icase_unique(names)
if len(names) == 0:
return []
@ -175,6 +178,24 @@ def get_tags_by_names(names: List[str]) -> List[model.Tag]:
)
def get_tags_by_exact_names(names: List[str]) -> List[model.Tag]:
"""
Returns a list of tags matching the names from the input list
"""
entries = []
if len(names) == 0:
return []
names = [name.lower() for name in names]
entries = (
db.session.query(model.Tag)
.join(model.TagName)
.filter(
sa.func.lower(model.TagName.name).in_(names)
)
.all())
return entries
def get_or_create_tags_by_names(
names: List[str],
) -> Tuple[List[model.Tag], List[model.Tag]]:

View File

@ -1,3 +1,4 @@
import copy
import re
from datetime import datetime
from typing import Any, Callable, Dict, List, Optional, Union
@ -5,7 +6,7 @@ from typing import Any, Callable, Dict, List, Optional, Union
import sqlalchemy as sa
from szurubooru import config, db, errors, model, rest
from szurubooru.func import auth, files, images, serialization, util
from szurubooru.func import auth, files, images, serialization, util, tags
class UserNotFoundError(errors.NotFoundError):
@ -107,6 +108,7 @@ class UserSerializer(serialization.BaseSerializer):
"lastLoginTime": self.serialize_last_login_time,
"version": self.serialize_version,
"rank": self.serialize_rank,
"blocklist": self.serialize_blocklist,
"avatarStyle": self.serialize_avatar_style,
"avatarUrl": self.serialize_avatar_url,
"commentCount": self.serialize_comment_count,
@ -138,6 +140,9 @@ class UserSerializer(serialization.BaseSerializer):
def serialize_avatar_url(self) -> Any:
return get_avatar_url(self.user)
def serialize_blocklist(self) -> Any:
return [tags.serialize_tag(tag) for tag in get_blocklist_tag_from_user(self.user)]
def serialize_comment_count(self) -> Any:
return self.user.comment_count
@ -294,6 +299,66 @@ def update_user_rank(
user.rank = rank
def get_blocklist_from_user(user: model.User) -> List[model.UserTagBlocklist]:
"""
Return the UserTagBlocklist objects related to given user
"""
rez = (db.session.query(model.UserTagBlocklist)
.filter(
model.UserTagBlocklist.user_id == user.user_id
)
.all())
return rez
def get_blocklist_tag_from_user(user: model.User) -> List[model.UserTagBlocklist]:
"""
Return the Tags blocklisted by given user
"""
rez = (db.session.query(model.UserTagBlocklist.tag_id)
.filter(
model.UserTagBlocklist.user_id == user.user_id
))
rez2 = (db.session.query(model.Tag)
.filter(
model.Tag.tag_id.in_(rez)
).all())
return rez2
def update_user_blocklist(user: model.User, new_blocklist_tags: Optional[List[model.Tag]]) -> List[List[model.UserTagBlocklist]]:
"""
Modify blocklist for given user.
If new_blocklist_tags is None, set the blocklist to configured default tag blocklist.
"""
assert user
to_add: List[model.UserTagBlocklist] = []
to_remove: List[model.UserTagBlocklist] = []
if new_blocklist_tags is None: # We're creating the user, use default config blocklist
if 'default_tag_blocklist' in config.config.keys():
for e in tags.get_tags_by_exact_names(config.config['default_tag_blocklist'].split(' ')):
to_add.append(model.UserTagBlocklist(user_id=user.user_id, tag_id=e.tag_id))
else:
new_blocklist_ids: List[int] = [e.tag_id for e in new_blocklist_tags]
previous_blocklist_tags: List[model.Tag] = get_blocklist_from_user(user)
previous_blocklist_ids: List[int] = [e.tag_id for e in previous_blocklist_tags]
original_previous_blocklist_ids = copy.copy(previous_blocklist_ids)
## Remove tags no longer in the new list
for i in range(len(original_previous_blocklist_ids)):
old_tag_id = original_previous_blocklist_ids[i]
if old_tag_id not in new_blocklist_ids:
to_remove.append(previous_blocklist_tags[i])
previous_blocklist_ids.remove(old_tag_id)
## Add tags not yet in the original list
for new_tag_id in new_blocklist_ids:
if new_tag_id not in previous_blocklist_ids:
to_add.append(model.UserTagBlocklist(user_id=user.user_id, tag_id=new_tag_id))
return to_add, to_remove
def update_user_avatar(
user: model.User, avatar_style: str, avatar_content: Optional[bytes] = None
) -> None:

View File

@ -0,0 +1,30 @@
'''
Add blocklist related fields
add_blocklist
Revision ID: 9ba5e3a6ee7c
Created at: 2023-05-20 22:28:10.824954
'''
import sqlalchemy as sa
from alembic import op
revision = '9ba5e3a6ee7c'
down_revision = 'adcd63ff76a2'
branch_labels = None
depends_on = None
def upgrade():
op.create_table(
"user_tag_blocklist",
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("tag_id", sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(["user_id"], ["user.id"]),
sa.ForeignKeyConstraint(["tag_id"], ["tag.id"]),
sa.PrimaryKeyConstraint("user_id", "tag_id"),
)
def downgrade():
op.drop_table('user_tag_blocklist')

View File

@ -16,4 +16,4 @@ from szurubooru.model.post import (
from szurubooru.model.snapshot import Snapshot
from szurubooru.model.tag import Tag, TagImplication, TagName, TagSuggestion
from szurubooru.model.tag_category import TagCategory
from szurubooru.model.user import User, UserToken
from szurubooru.model.user import UserTagBlocklist, User, UserToken

View File

@ -5,6 +5,46 @@ from szurubooru.model.comment import Comment
from szurubooru.model.post import Post, PostFavorite, PostScore
class UserTagBlocklist(Base):
__tablename__ = "user_tag_blocklist"
user_id = sa.Column(
"user_id",
sa.Integer,
sa.ForeignKey("user.id"),
primary_key=True,
nullable=False,
index=True,
)
tag_id = sa.Column(
"tag_id",
sa.Integer,
sa.ForeignKey("tag.id"),
primary_key=True,
nullable=False,
index=True,
)
tag = sa.orm.relationship(
"Tag",
backref=sa.orm.backref("user_tag_blocklist", cascade="all, delete-orphan"),
)
user = sa.orm.relationship(
"User",
backref=sa.orm.backref("user_tag_blocklist", cascade="all, delete-orphan"),
)
def __init__(self, user_id: int=None, tag_id: int=None, user=None, tag=None) -> None:
if user_id is not None:
self.user_id = user_id
if tag_id is not None:
self.tag_id = tag_id
if user is not None:
self.user = user
if tag is not None:
self.tag = tag
class User(Base):
__tablename__ = "user"
@ -35,6 +75,7 @@ class User(Base):
"avatar_style", sa.Unicode(32), nullable=False, default=AVATAR_GRAVATAR
)
blocklist = sa.orm.relationship("UserTagBlocklist")
comments = sa.orm.relationship("Comment")
@property

View File

@ -2,9 +2,9 @@ from typing import Any, Dict, Optional, Tuple
import sqlalchemy as sa
from szurubooru import db, errors, model
from szurubooru.func import util
from szurubooru.search import criteria, tokens
from szurubooru import config, db, errors, model
from szurubooru.func import tags, users, util
from szurubooru.search import criteria, parser, tokens
from szurubooru.search.configs import util as search_util
from szurubooru.search.configs.base_search_config import (
BaseSearchConfig,
@ -178,6 +178,32 @@ class PostSearchConfig(BaseSearchConfig):
new_special_tokens.append(token)
search_query.special_tokens = new_special_tokens
blocklist_to_use = ""
if self.user: # Ensure there's a user object
if (self.user.rank == model.User.RANK_ANONYMOUS) and config.config["default_tag_blocklist_for_anonymous"]:
# Anonymous user, if configured to use default blocklist, do so
blocklist_to_use = config.config["default_tag_blocklist"]
else:
# Registered user, use their blocklist
user_blocklist_tags = users.get_blocklist_tag_from_user(self.user)
if user_blocklist_tags:
user_blocklist = db.session.query(model.Tag.first_name).filter(
model.Tag.tag_id.in_([e.tag_id for e in user_blocklist_tags])
).all()
blocklist_to_use = [e[0] for e in user_blocklist]
blocklist_to_use = " ".join(blocklist_to_use)
if len(blocklist_to_use) > 0:
# TODO Sort an already parsed and checked version instead?
blocklist_query = parser.Parser().parse(blocklist_to_use)
search_query_orig_list = [e.criterion.original_text for e in search_query.anonymous_tokens]
for t in blocklist_query.anonymous_tokens:
if t.criterion.original_text in search_query_orig_list:
continue
t.negated = True
search_query.anonymous_tokens.append(t)
def create_around_query(self) -> SaQuery:
return db.session.query(model.Post).options(sa.orm.lazyload("*"))

View File

@ -26,6 +26,8 @@ def test_info_api(
"tag_name_regex": "3",
"tag_category_name_regex": "4",
"default_rank": "5",
"default_tag_blocklist": "testTag",
"default_tag_blocklist_for_anonymous": True,
"privileges": {
"test_key1": "test_value1",
"test_key2": "test_value2",
@ -48,6 +50,8 @@ def test_info_api(
"tagNameRegex": "3",
"tagCategoryNameRegex": "4",
"defaultUserRank": "5",
"defaultTagBlocklist": "testTag",
"defaultTagBlocklistForAnonymous": True,
"privileges": {
"testKey1": "test_value1",
"testKey2": "test_value2",

View File

@ -0,0 +1,139 @@
from datetime import datetime
from unittest.mock import patch
import pytest
from szurubooru import api, db, errors, model
from szurubooru.func import posts
## TODO: Add following tests:
## - Retrieve posts without blocklist active for current registered user
## - Retrieve posts with blocklist active for current registered user
## - Retrieve posts without blocklist active for anonymous user
## - Retrieve posts with blocklist active for anonymous user
## - Creation of user with default blocklist (test that user_blocklist entries are properly added to db, with right infos)
## - Modification of user with/without blocklist changes
## - Retrieve posts with a query including a blocklisted tag (it should include results with the tag)
## - Behavior when creating user with default blocklist and tags from this list don't exist (blocklist entry shouldn't be added)
## - Test all small functions used across blocklist features
def test_blocklist(user_factory, post_factory, context_factory, config_injector, user_blocklist_factory, tag_factory):
"""
Test that user blocklist is applied on post retrieval
"""
tag1 = tag_factory(names=['tag1'])
tag2 = tag_factory(names=['tag2'])
tag3 = tag_factory(names=['tag3'])
post1 = post_factory(id=11, tags=[tag1, tag2])
post2 = post_factory(id=12, tags=[tag1])
post3 = post_factory(id=13, tags=[tag2])
post4 = post_factory(id=14, tags=[tag3])
post5 = post_factory(id=15)
user1 = user_factory(rank=model.User.RANK_REGULAR)
blocklist1 = user_blocklist_factory(tag=tag1, user=user1)
config_injector({
"privileges": {
"posts:list": model.User.RANK_REGULAR,
}
})
db.session.add_all([tag1, tag2, tag3, user1, blocklist1, post1, post2, post3, post4, post5])
db.session.flush()
# We can't check that the posts we retrieve are the ones we want
with patch("szurubooru.func.posts.serialize_post"):
posts.serialize_post.side_effect = (
lambda post, *_args, **_kwargs: "serialized post %d" % post.post_id
)
result = api.post_api.get_posts(
context_factory(
params={"query": "", "offset": 0},
user=user1,
)
)
assert result == {
"query": "",
"offset": 0,
"limit": 100,
"total": 3,
"results": ["serialized post 15", "serialized post 14", "serialized post 13"],
}
# def test_blocklist_no_anonymous(user_factory, post_factory, context_factory, config_injector, tag_factory):
# """
# Test that default blocklist isn't applied on anonymous users on post retrieval if disabled in configuration
# """
# tag1 = tag_factory(names=['tag1'])
# post1 = post_factory(id=21, tags=[tag1])
# post2 = post_factory(id=22, tags=[tag1])
# post3 = post_factory(id=23)
# user1 = user_factory(rank=model.User.RANK_ANONYMOUS)
# config_injector({
# "default_tag_blocklist": "tag1",
# "default_tag_blocklist_for_anonymous": False,
# "privileges": {
# "posts:list": model.User.RANK_ANONYMOUS,
# }
# })
# db.session.add_all([tag1, post1, post2, post3])
# db.session.flush()
# with patch("szurubooru.func.posts.serialize_post"):
# posts.serialize_post.side_effect = (
# lambda post, *_args, **_kwargs: "serialized post %d" % post.post_id
# )
# result = api.post_api.get_posts(
# context_factory(
# params={"query": "", "offset": 0},
# user=user1,
# )
# )
# assert result == {
# "query": "",
# "offset": 0,
# "limit": 100,
# "total": 3,
# "results": ["serialized post 23", "serialized post 22", "serialized post 21"],
# }
def test_blocklist_anonymous(user_factory, post_factory, context_factory, config_injector, tag_factory):
"""
Test that default blocklist is applied on anonymous users on post retrieval if enabled in configuration
"""
tag1 = tag_factory(names=['tag1'])
tag2 = tag_factory(names=['tag2'])
tag3 = tag_factory(names=['tag3'])
post1 = post_factory(id=31, tags=[tag1, tag2])
post2 = post_factory(id=32, tags=[tag1])
post3 = post_factory(id=33, tags=[tag2])
post4 = post_factory(id=34, tags=[tag3])
post5 = post_factory(id=35)
config_injector({
"default_tag_blocklist": "tag3",
"default_tag_blocklist_for_anonymous": True,
"privileges": {
"posts:list": model.User.RANK_ANONYMOUS,
}
})
db.session.add_all([tag1, tag2, tag3, post1, post2, post3, post4, post5])
db.session.flush()
with patch("szurubooru.func.posts.serialize_post"):
posts.serialize_post.side_effect = (
lambda post, *_args, **_kwargs: "serialized post %d" % post.post_id
)
result = api.post_api.get_posts(
context_factory(
params={"query": "", "offset": 0},
user=user_factory(rank=model.User.RANK_ANONYMOUS),
)
)
assert result == {
"query": "",
"offset": 0,
"limit": 100,
"total": 4,
"results": ["serialized post 35", "serialized post 33", "serialized post 32", "serialized post 31"],
}
## TODO: Test when we add blocklist items to the query

View File

@ -21,6 +21,8 @@ def test_creating_user(user_factory, context_factory, fake_datetime):
"szurubooru.func.users.update_user_rank"
), patch(
"szurubooru.func.users.update_user_avatar"
), patch(
"szurubooru.func.users.update_user_blocklist"
), patch(
"szurubooru.func.users.serialize_user"
), fake_datetime(
@ -28,6 +30,7 @@ def test_creating_user(user_factory, context_factory, fake_datetime):
):
users.serialize_user.return_value = "serialized user"
users.create_user.return_value = user
users.update_user_blocklist.return_value = ([],[])
result = api.user_api.create_user(
context_factory(
params={
@ -50,6 +53,7 @@ def test_creating_user(user_factory, context_factory, fake_datetime):
assert not users.update_user_email.called
users.update_user_rank.called_once_with(user, "moderator")
users.update_user_avatar.called_once_with(user, "manual", b"...")
users.update_user_blocklist.called_once_with(user, None)
@pytest.mark.parametrize("field", ["name", "password"])

View File

@ -14,11 +14,13 @@ def inject_config(config_injector):
"users:edit:self:name": model.User.RANK_REGULAR,
"users:edit:self:pass": model.User.RANK_REGULAR,
"users:edit:self:email": model.User.RANK_REGULAR,
"users:edit:self:blocklist": model.User.RANK_REGULAR,
"users:edit:self:rank": model.User.RANK_MODERATOR,
"users:edit:self:avatar": model.User.RANK_MODERATOR,
"users:edit:any:name": model.User.RANK_MODERATOR,
"users:edit:any:pass": model.User.RANK_MODERATOR,
"users:edit:any:email": model.User.RANK_MODERATOR,
"users:edit:any:blocklist": model.User.RANK_MODERATOR,
"users:edit:any:rank": model.User.RANK_ADMINISTRATOR,
"users:edit:any:avatar": model.User.RANK_ADMINISTRATOR,
},

View File

@ -136,6 +136,20 @@ def user_token_factory(user_factory):
return factory
@pytest.fixture
def user_blocklist_factory(user_factory, tag_factory):
def factory(tag=None, user=None):
if user is None:
user = user_factory()
if tag is None:
tag = tag_factory()
return model.UserTagBlocklist(
tag=tag, user=user
)
return factory
@pytest.fixture
def tag_category_factory():
def factory(name=None, color="dummy", order=1, default=False):
@ -172,6 +186,7 @@ def post_factory():
id=None,
safety=model.Post.SAFETY_SAFE,
type=model.Post.TYPE_IMAGE,
tags=[],
checksum="...",
):
post = model.Post()
@ -182,6 +197,7 @@ def post_factory():
post.flags = []
post.mime_type = "application/octet-stream"
post.creation_time = datetime(1996, 1, 1)
post.tags = tags
return post
return factory

View File

@ -158,6 +158,7 @@ def test_serialize_user(user_factory):
"avatarUrl": "https://example.com/avatar.png",
"likedPostCount": 66,
"dislikedPostCount": 33,
"blocklist": [],
"commentCount": 0,
"favoritePostCount": 0,
"uploadedPostCount": 0,
@ -235,7 +236,7 @@ def test_create_user_for_first_user(fake_datetime):
"szurubooru.func.users.update_user_password"
), patch("szurubooru.func.users.update_user_email"), fake_datetime(
"1997-01-01"
):
), patch("szurubooru.func.users.update_user_blocklist"):
user = users.create_user("name", "password", "email")
assert user.creation_time == datetime(1997, 1, 1)
assert user.last_login_time is None
@ -251,7 +252,8 @@ def test_create_user_for_subsequent_users(user_factory, config_injector):
db.session.flush()
with patch("szurubooru.func.users.update_user_name"), patch(
"szurubooru.func.users.update_user_email"
), patch("szurubooru.func.users.update_user_password"):
), patch("szurubooru.func.users.update_user_password"
), patch("szurubooru.func.users.update_user_blocklist"):
user = users.create_user("name", "password", "email")
assert user.rank == model.User.RANK_REGULAR