diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c06689a2..ce2bd780 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,49 +1,46 @@ ## How to contribute to Calibre-Web -First of all, we would like to thank you for reading this text. we are happy you are willing to contribute to Calibre-Web +First of all, we would like to thank you for reading this text. We are happy you are willing to contribute to Calibre-Web. ### **General** -**Communication language** is english. Google translated texts are not as bad as you might think, they are usually understandable, so don't worry if you generate your post that way. +**Communication language** is English. Google translated texts are not as bad as you might think, they are usually understandable, so don't worry if you generate your post that way. **Calibre-Web** is not **Calibre**. If you are having a question regarding Calibre please post this at their [repository](https://github.com/kovidgoyal/calibre). -**Docker-Containers** of Calibre-Web are maintained by different persons than the people who drive Calibre-Web. If we come to the conclusion during our analysis that the problem is related to Docker, we might ask you to open a new issue at the reprository of the Docker Container. +**Docker-Containers** of Calibre-Web are maintained by different persons than the people who drive Calibre-Web. If we come to the conclusion during our analysis that the problem is related to Docker, we might ask you to open a new issue at the repository of the Docker Container. -If you are having **Basic Installation Problems** with python or it's dependencies, please consider using your favorite search engine to find a solution. In case you can't find a solution, we are happy to help you. +If you are having **Basic Installation Problems** with python or its dependencies, please consider using your favorite search engine to find a solution. In case you can't find a solution, we are happy to help you. -We can offer only very limited support regarding configuration of **Reverse-Proxy Installations**, **OPDS-Reader** or other programs in combination with Calibre-Web. +We can offer only very limited support regarding configuration of **Reverse-Proxy Installations**, **OPDS-Reader** or other programs in combination with Calibre-Web. ### **Translation** -Some of the user languages in Calibre-Web having missing translations. We are happy to add the missing texts if you translate them. Create a Pull Request, create an issue with the .po file attached, or write an email to "ozzie.fernandez.isaacs@googlemail.com" with attached translation file. To display all book languages in your native language an additional file is used (iso_language_names.py). The content of this file is autogenerated with the corresponding translations of Calibre, please do not edit this file on your own. +Some of the user languages in Calibre-Web having missing translations. We are happy to add the missing texts if you translate them. Create a Pull Request, create an issue with the .po file attached, or write an email to "ozzie.fernandez.isaacs@googlemail.com" with attached translation file. To display all book languages in your native language an additional file is used (iso_language_names.py). The content of this file is auto-generated with the corresponding translations of Calibre, please do not edit this file on your own. ### **Documentation** -The Calibre-Web documentation is hosted in the Github [Wiki](https://github.com/janeczku/calibre-web/wiki). The Wiki is open to everybody, if you find a problem, feel free to correct it. If information is missing, you are welcome to add it. The content will be reviewed time by time. Please try to be consitent with the form with the other Wiki pages (e.g. the project name is Calibre-Web with 2 capital letters and a dash in between). +The Calibre-Web documentation is hosted in the Github [Wiki](https://github.com/janeczku/calibre-web/wiki). The Wiki is open to everybody, if you find a problem, feel free to correct it. If information is missing, you are welcome to add it. The content will be reviewed time by time. Please try to be consistent with the form with the other Wiki pages (e.g. the project name is Calibre-Web with 2 capital letters and a dash in between). ### **Reporting a bug** -Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Please write intead an email to "ozzie.fernandez.isaacs@googlemail.com". +Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Instead, please write an email to "ozzie.fernandez.isaacs@googlemail.com". Ensure the ***bug was not already reported** by searching on GitHub under [Issues](https://github.com/janeczku/calibre-web/issues). Please also check if a solution for your problem can be found in the [wiki](https://github.com/janeczku/calibre-web/wiki). -If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=bug_report.md&title=). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue. +If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=bug_report.md&title=). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue. ### **Feature Request** -If there is a feature missing in Calibre-Web and you can't find a feature request in the [Issues](https://github.com/janeczku/calibre-web/issues) section, you could create a [feature request](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=feature_request.md&title=). -We will not extend Calibre-Web with any more login abilitys or add further files storages, or file syncing ability. Furthermore Calibre-Web is made for home usage for company inhouse usage, so requests regarding any sorts of social interaction capability, payment routines, search engine or web site analytics integration will not be implemeted. +If there is a feature missing in Calibre-Web and you can't find a feature request in the [Issues](https://github.com/janeczku/calibre-web/issues) section, you could create a [feature request](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=feature_request.md&title=). +We will not extend Calibre-Web with any more login abilities or add further files storages, or file syncing ability. Furthermore Calibre-Web is made for home usage for company in-house usage, so requests regarding any sorts of social interaction capability, payment routines, search engine or web site analytics integration will not be implemented. ### **Contributing code to Calibre-Web** Open a new GitHub pull request with the patch. Ensure the PR description clearly describes the problem and solution. Include the relevant issue number if applicable. -In case your code enhances features of Calibre-Web: Create your pull request for the development branch if your enhancement consits of more than some lines of code in a local section of Calibre-Webs code. This makes it easier to test it and check all implication before it's made public. - -Please check if your code runs on Python 2.7 (still necessary in 2020) and mainly on python 3. If possible and the feature is related to operating system functions, try to check it on Windows and Linux. -Calibre-Web is automatically tested on Linux in combination with python 3.7. The code for testing is in a [seperate repo](https://github.com/OzzieIsaacs/calibre-web-test) on Github. It uses unittest and performs real system tests with selenium, would be great if you could consider also writing some tests. -A static code analysis is done by Codacy, but it's partly broken and doesn't run automatically. You could check your code with ESLint before contributing, a configuration file can be found in the projects root folder. - - +In case your code enhances features of Calibre-Web: Create your pull request for the development branch if your enhancement consists of more than some lines of code in a local section of Calibre-Webs code. This makes it easier to test it and check all implication before it's made public. +Please check if your code runs on Python 2.7 (still necessary in 2020) and mainly on python 3. If possible and the feature is related to operating system functions, try to check it on Windows and Linux. +Calibre-Web is automatically tested on Linux in combination with python 3.7. The code for testing is in a [separate repo](https://github.com/OzzieIsaacs/calibre-web-test) on Github. It uses unit tests and performs real system tests with selenium; it would be great if you could consider also writing some tests. +A static code analysis is done by Codacy, but it's partly broken and doesn't run automatically. You could check your code with ESLint before contributing, a configuration file can be found in the projects root folder. diff --git a/cps/admin.py b/cps/admin.py index 4fe027a2..5286cf8e 100644 --- a/cps/admin.py +++ b/cps/admin.py @@ -99,7 +99,7 @@ def shutdown(): if task == 2: log.warning("reconnecting to calibre database") - calibre_db.setup_db(config, ub.app_DB_path) + calibre_db.reconnect_db(config, ub.app_DB_path) showtext['text'] = _(u'Reconnect successful') return json.dumps(showtext) diff --git a/cps/config_sql.py b/cps/config_sql.py index 1135516d..3885bb01 100644 --- a/cps/config_sql.py +++ b/cps/config_sql.py @@ -22,7 +22,7 @@ import os import json import sys -from sqlalchemy import exc, Column, String, Integer, SmallInteger, Boolean, BLOB +from sqlalchemy import exc, Column, String, Integer, SmallInteger, Boolean, BLOB, JSON from sqlalchemy.ext.declarative import declarative_base from . import constants, cli, logger, ub @@ -92,7 +92,7 @@ class _Settings(_Base): config_use_google_drive = Column(Boolean, default=False) config_google_drive_folder = Column(String) - config_google_drive_watch_changes_response = Column(String) + config_google_drive_watch_changes_response = Column(JSON, default={}) config_use_goodreads = Column(Boolean, default=False) config_goodreads_api_key = Column(String) @@ -102,7 +102,6 @@ class _Settings(_Base): config_kobo_proxy = Column(Boolean, default=False) - config_ldap_provider_url = Column(String, default='example.org') config_ldap_port = Column(SmallInteger, default=389) config_ldap_authentication = Column(SmallInteger, default=constants.LDAP_AUTH_SIMPLE) @@ -215,20 +214,20 @@ class _ConfigSQL(object): return self.show_element_new_user(constants.DETAIL_RANDOM) def list_denied_tags(self): - mct = self.config_denied_tags.split(",") - return [t.strip() for t in mct] + mct = self.config_denied_tags or "" + return [t.strip() for t in mct.split(",")] def list_allowed_tags(self): - mct = self.config_allowed_tags.split(",") - return [t.strip() for t in mct] + mct = self.config_allowed_tags or "" + return [t.strip() for t in mct.split(",")] def list_denied_column_values(self): - mct = self.config_denied_column_value.split(",") - return [t.strip() for t in mct] + mct = self.config_denied_column_value or "" + return [t.strip() for t in mct.split(",")] def list_allowed_column_values(self): - mct = self.config_allowed_column_value.split(",") - return [t.strip() for t in mct] + mct = self.config_allowed_column_value or "" + return [t.strip() for t in mct.split(",")] def get_log_level(self): return logger.get_level_name(self.config_log_level) @@ -281,10 +280,6 @@ class _ConfigSQL(object): v = column.default.arg setattr(self, k, v) - if self.config_google_drive_watch_changes_response: - self.config_google_drive_watch_changes_response = \ - json.loads(self.config_google_drive_watch_changes_response) - have_metadata_db = bool(self.config_calibre_dir) if have_metadata_db: if not self.config_use_google_drive: @@ -303,10 +298,6 @@ class _ConfigSQL(object): '''Apply all configuration values to the underlying storage.''' s = self._read_from_storage() # type: _Settings - if self.config_google_drive_watch_changes_response: - self.config_google_drive_watch_changes_response = json.dumps( - self.config_google_drive_watch_changes_response) - for k, v in self.__dict__.items(): if k[0] == '_': continue @@ -361,10 +352,10 @@ def _migrate_table(session, orm_class): def autodetect_calibre_binary(): if sys.platform == "win32": - calibre_path = ["C:\\program files\calibre\ebook-convert.exe", - "C:\\program files(x86)\calibre\ebook-convert.exe", - "C:\\program files(x86)\calibre2\ebook-convert.exe", - "C:\\program files\calibre2\ebook-convert.exe"] + calibre_path = ["C:\\program files\\calibre\\ebook-convert.exe", + "C:\\program files(x86)\\calibre\\ebook-convert.exe", + "C:\\program files(x86)\\calibre2\\ebook-convert.exe", + "C:\\program files\\calibre2\\ebook-convert.exe"] else: calibre_path = ["/opt/calibre/ebook-convert"] for element in calibre_path: diff --git a/cps/db.py b/cps/db.py index 0ed89467..9d06b149 100644 --- a/cps/db.py +++ b/cps/db.py @@ -33,8 +33,10 @@ from sqlalchemy.orm import relationship, sessionmaker, scoped_session from sqlalchemy.orm.collections import InstrumentedList from sqlalchemy.ext.declarative import declarative_base, DeclarativeMeta from sqlalchemy.exc import OperationalError +from sqlalchemy.pool import StaticPool from flask_login import current_user from sqlalchemy.sql.expression import and_, true, false, text, func, or_ +from sqlalchemy.ext.associationproxy import association_proxy from babel import Locale as LC from babel.core import UnknownLocaleError from flask_babel import gettext as _ @@ -312,7 +314,7 @@ class Books(Base): flags = Column(Integer, nullable=False, default=1) authors = relationship('Authors', secondary=books_authors_link, backref='books') - tags = relationship('Tags', secondary=books_tags_link, backref='books',order_by="Tags.name") + tags = relationship('Tags', secondary=books_tags_link, backref='books', order_by="Tags.name") comments = relationship('Comments', backref='books') data = relationship('Data', backref='books') series = relationship('Series', secondary=books_series_link, backref='books') @@ -438,7 +440,6 @@ class CalibreDB(threading.Thread): def setup_db(self, config, app_db_path): self.config = config self.dispose() - # global engine if not config.config_calibre_dir: config.invalidate() @@ -450,11 +451,11 @@ class CalibreDB(threading.Thread): return False try: - #engine = create_engine('sqlite:///{0}'.format(dbpath), self.engine = create_engine('sqlite://', echo=False, isolation_level="SERIALIZABLE", - connect_args={'check_same_thread': False}) + connect_args={'check_same_thread': False}, + poolclass=StaticPool) self.engine.execute("attach database '{}' as calibre;".format(dbpath)) self.engine.execute("attach database '{}' as app_settings;".format(app_db_path)) @@ -474,34 +475,46 @@ class CalibreDB(threading.Thread): books_custom_column_links = {} for row in cc: if row.datatype not in cc_exceptions: - books_custom_column_links[row.id] = Table('books_custom_column_' + str(row.id) + '_link', Base.metadata, - Column('book', Integer, ForeignKey('books.id'), - primary_key=True), - Column('value', Integer, - ForeignKey('custom_column_' + str(row.id) + '.id'), - primary_key=True) - ) + if row.datatype == 'series': + dicttable = {'__tablename__': 'books_custom_column_' + str(row.id) + '_link', + 'id': Column(Integer, primary_key=True), + 'book': Column(Integer, ForeignKey('books.id'), + primary_key=True), + 'map_value': Column('value', Integer, + ForeignKey('custom_column_' + + str(row.id) + '.id'), + primary_key=True), + 'extra': Column(Float), + 'asoc' : relationship('custom_column_' + str(row.id), uselist=False), + 'value' : association_proxy('asoc', 'value') + } + books_custom_column_links[row.id] = type(str('books_custom_column_' + str(row.id) + '_link'), + (Base,), dicttable) + else: + books_custom_column_links[row.id] = Table('books_custom_column_' + str(row.id) + '_link', + Base.metadata, + Column('book', Integer, ForeignKey('books.id'), + primary_key=True), + Column('value', Integer, + ForeignKey('custom_column_' + + str(row.id) + '.id'), + primary_key=True) + ) cc_ids.append([row.id, row.datatype]) - if row.datatype == 'bool': - ccdict = {'__tablename__': 'custom_column_' + str(row.id), - 'id': Column(Integer, primary_key=True), - 'book': Column(Integer, ForeignKey('books.id')), - 'value': Column(Boolean)} + + ccdict = {'__tablename__': 'custom_column_' + str(row.id), + 'id': Column(Integer, primary_key=True)} + if row.datatype == 'float': + ccdict['value'] = Column(Float) elif row.datatype == 'int': - ccdict = {'__tablename__': 'custom_column_' + str(row.id), - 'id': Column(Integer, primary_key=True), - 'book': Column(Integer, ForeignKey('books.id')), - 'value': Column(Integer)} - elif row.datatype == 'float': - ccdict = {'__tablename__': 'custom_column_' + str(row.id), - 'id': Column(Integer, primary_key=True), - 'book': Column(Integer, ForeignKey('books.id')), - 'value': Column(Float)} + ccdict['value'] = Column(Integer) + elif row.datatype == 'bool': + ccdict['value'] = Column(Boolean) else: - ccdict = {'__tablename__': 'custom_column_' + str(row.id), - 'id': Column(Integer, primary_key=True), - 'value': Column(String)} - cc_classes[row.id] = type(str('Custom_Column_' + str(row.id)), (Base,), ccdict) + ccdict['value'] = Column(String) + if row.datatype in ['float', 'int', 'bool']: + ccdict['book'] = Column(Integer, ForeignKey('books.id')) + cc_classes[row.id] = type(str('custom_column_' + str(row.id)), (Base,), ccdict) for cc_id in cc_ids: if (cc_id[1] == 'bool') or (cc_id[1] == 'int') or (cc_id[1] == 'float'): @@ -511,6 +524,11 @@ class CalibreDB(threading.Thread): primaryjoin=( Books.id == cc_classes[cc_id[0]].book), backref='books')) + elif (cc_id[1] == 'series'): + setattr(Books, + 'custom_column_' + str(cc_id[0]), + relationship(books_custom_column_links[cc_id[0]], + backref='books')) else: setattr(Books, 'custom_column_' + str(cc_id[0]), diff --git a/cps/editbooks.py b/cps/editbooks.py index 0fe2d781..48e4fe23 100644 --- a/cps/editbooks.py +++ b/cps/editbooks.py @@ -867,8 +867,8 @@ def upload(): # move cover to final directory, including book id if has_cover: + new_coverpath = os.path.join(config.config_calibre_dir, db_book.path, "cover.jpg") try: - new_coverpath = os.path.join(config.config_calibre_dir, db_book.path, "cover.jpg") copyfile(meta.cover, new_coverpath) os.unlink(meta.cover) except OSError as e: diff --git a/cps/gdrive.py b/cps/gdrive.py index aa3743d2..82a19890 100644 --- a/cps/gdrive.py +++ b/cps/gdrive.py @@ -34,18 +34,17 @@ from flask import Blueprint, flash, request, redirect, url_for, abort from flask_babel import gettext as _ from flask_login import login_required -try: - from googleapiclient.errors import HttpError -except ImportError: - pass - from . import logger, gdriveutils, config, ub, calibre_db from .web import admin_required - gdrive = Blueprint('gdrive', __name__) log = logger.create() +try: + from googleapiclient.errors import HttpError +except ImportError as err: + log.debug(("Cannot import googleapiclient, using gdrive will not work: %s", err)) + current_milli_time = lambda: int(round(time() * 1000)) gdrive_watch_callback_token = 'target=calibreweb-watch_files' @@ -73,7 +72,7 @@ def google_drive_callback(): credentials = gdriveutils.Gauth.Instance().auth.flow.step2_exchange(auth_code) with open(gdriveutils.CREDENTIALS, 'w') as f: f.write(credentials.to_json()) - except ValueError as error: + except (ValueError, AttributeError) as error: log.error(error) return redirect(url_for('admin.configuration')) @@ -94,8 +93,7 @@ def watch_gdrive(): try: result = gdriveutils.watchChange(gdriveutils.Gdrive.Instance().drive, notification_id, 'web_hook', address, gdrive_watch_callback_token, current_milli_time() + 604800*1000) - config.config_google_drive_watch_changes_response = json.dumps(result) - # after save(), config_google_drive_watch_changes_response will be a json object, not string + config.config_google_drive_watch_changes_response = result config.save() except HttpError as e: reason=json.loads(e.content)['error']['errors'][0] @@ -118,7 +116,7 @@ def revoke_watch_gdrive(): last_watch_response['resourceId']) except HttpError: pass - config.config_google_drive_watch_changes_response = None + config.config_google_drive_watch_changes_response = {} config.save() return redirect(url_for('admin.configuration')) @@ -155,7 +153,7 @@ def on_received_watch_confirmation(): log.info('Setting up new DB') # prevent error on windows, as os.rename does on exisiting files move(os.path.join(tmpDir, "tmp_metadata.db"), dbpath) - calibre_db.setup_db(config, ub.app_DB_path) + calibre_db.reconnect_db(config, ub.app_DB_path) except Exception as e: log.exception(e) updateMetaData() diff --git a/cps/gdriveutils.py b/cps/gdriveutils.py index 9ea0479d..a996f879 100644 --- a/cps/gdriveutils.py +++ b/cps/gdriveutils.py @@ -36,7 +36,9 @@ try: from apiclient import errors from httplib2 import ServerNotFoundError gdrive_support = True -except ImportError: + importError = None +except ImportError as err: + importError = err gdrive_support = False from . import logger, cli, config @@ -52,6 +54,8 @@ if gdrive_support: logger.get('googleapiclient.discovery_cache').setLevel(logger.logging.ERROR) if not logger.is_debug_enabled(): logger.get('googleapiclient.discovery').setLevel(logger.logging.ERROR) +else: + log.debug("Cannot import pydrive,httplib2, using gdrive will not work: %s", importError) class Singleton: @@ -99,7 +103,11 @@ class Singleton: @Singleton class Gauth: def __init__(self): - self.auth = GoogleAuth(settings_file=SETTINGS_YAML) + try: + self.auth = GoogleAuth(settings_file=SETTINGS_YAML) + except NameError as error: + log.error(error) + self.auth = None @Singleton @@ -594,8 +602,12 @@ def get_error_text(client_secrets=None): if not os.path.isfile(CLIENT_SECRETS): return 'client_secrets.json is missing or not readable' - with open(CLIENT_SECRETS, 'r') as settings: - filedata = json.load(settings) + try: + with open(CLIENT_SECRETS, 'r') as settings: + filedata = json.load(settings) + except PermissionError: + return 'client_secrets.json is missing or not readable' + if 'web' not in filedata: return 'client_secrets.json is not configured for web application' if 'redirect_uris' not in filedata['web']: diff --git a/cps/helper.py b/cps/helper.py index 48efc69b..681719a9 100644 --- a/cps/helper.py +++ b/cps/helper.py @@ -295,15 +295,16 @@ def delete_book_file(book, calibrepath, book_format=None): return True, None else: if os.path.isdir(path): - if len(next(os.walk(path))[1]): - log.error("Deleting book %s failed, path has subfolders: %s", book.id, book.path) - return False , _("Deleting book %(id)s failed, path has subfolders: %(path)s", - id=book.id, - path=book.path) try: - for root, __, files in os.walk(path): + for root, folders, files in os.walk(path): for f in files: os.unlink(os.path.join(root, f)) + if len(folders): + log.warning("Deleting book {} failed, path {} has subfolders: {}".format(book.id, + book.path, folders)) + return True, _("Deleting bookfolder for book %(id)s failed, path has subfolders: %(path)s", + id=book.id, + path=book.path) shutil.rmtree(path) except (IOError, OSError) as e: log.error("Deleting book %s failed: %s", book.id, e) @@ -339,13 +340,13 @@ def update_dir_structure_file(book_id, calibrepath, first_author): new_title_path = os.path.join(os.path.dirname(path), new_titledir) try: if not os.path.exists(new_title_path): - os.renames(path, new_title_path) + os.renames(os.path.normcase(path), os.path.normcase(new_title_path)) else: log.info("Copying title: %s into existing: %s", path, new_title_path) for dir_name, __, file_list in os.walk(path): for file in file_list: - os.renames(os.path.join(dir_name, file), - os.path.join(new_title_path + dir_name[len(path):], file)) + os.renames(os.path.normcase(os.path.join(dir_name, file)), + os.path.normcase(os.path.join(new_title_path + dir_name[len(path):], file))) path = new_title_path localbook.path = localbook.path.split('/')[0] + '/' + new_titledir except OSError as ex: @@ -356,7 +357,7 @@ def update_dir_structure_file(book_id, calibrepath, first_author): if authordir != new_authordir: new_author_path = os.path.join(calibrepath, new_authordir, os.path.basename(path)) try: - os.renames(path, new_author_path) + os.renames(os.path.normcase(path), os.path.normcase(new_author_path)) localbook.path = new_authordir + '/' + localbook.path.split('/')[1] except OSError as ex: log.error("Rename author from: %s to %s: %s", path, new_author_path, ex) @@ -369,8 +370,9 @@ def update_dir_structure_file(book_id, calibrepath, first_author): new_name = get_valid_filename(localbook.title) + ' - ' + get_valid_filename(new_authordir) path_name = os.path.join(calibrepath, new_authordir, os.path.basename(path)) for file_format in localbook.data: - os.renames(os.path.join(path_name, file_format.name + '.' + file_format.format.lower()), - os.path.join(path_name, new_name + '.' + file_format.format.lower())) + os.renames(os.path.normcase( + os.path.join(path_name, file_format.name + '.' + file_format.format.lower())), + os.path.normcase(os.path.join(path_name, new_name + '.' + file_format.format.lower()))) file_format.name = new_name except OSError as ex: log.error("Rename file in path %s to %s: %s", path, new_name, ex) diff --git a/cps/jinjia.py b/cps/jinjia.py index b1b42939..a6df156d 100644 --- a/cps/jinjia.py +++ b/cps/jinjia.py @@ -107,3 +107,10 @@ def timestamptodate(date, fmt=None): @jinjia.app_template_filter('yesno') def yesno(value, yes, no): return yes if value else no + +@jinjia.app_template_filter('formatfloat') +def formatfloat(value, decimals=1): + formatedstring = '%d' % value + if (value % 1) != 0: + formatedstring = ('%s.%d' % (formatedstring, (value % 1) * 10**decimals)).rstrip('0') + return formatedstring diff --git a/cps/kobo.py b/cps/kobo.py index 97d55db0..70c4b45a 100644 --- a/cps/kobo.py +++ b/cps/kobo.py @@ -19,8 +19,6 @@ import base64 import datetime -import itertools -import json import sys import os import uuid @@ -267,7 +265,7 @@ def HandleMetadataRequest(book_uuid): def get_download_url_for_book(book, book_format): if not current_app.wsgi_app.is_proxied: - if ':' in request.host and not request.host.endswith(']') : + if ':' in request.host and not request.host.endswith(']'): host = "".join(request.host.split(':')[:-1]) else: host = request.host @@ -317,8 +315,15 @@ def get_description(book): # TODO handle multiple authors def get_author(book): if not book.authors: - return None - return book.authors[0].name + return {"Contributors": None} + if len(book.authors) > 1: + author_list = [] + autor_roles = [] + for author in book.authors: + autor_roles.append({"Name":author.name, "Role":"Author"}) + author_list.append(author.name) + return {"ContributorRoles": autor_roles, "Contributors":author_list} + return {"ContributorRoles": [{"Name":book.authors[0].name, "Role":"Author"}], "Contributors": book.authors[0].name} def get_publisher(book): @@ -357,7 +362,7 @@ def get_metadata(book): book_uuid = book.uuid metadata = { "Categories": ["00000000-0000-0000-0000-000000000001",], - "Contributors": get_author(book), + # "Contributors": get_author(book), "CoverImageId": book_uuid, "CrossRevisionId": book_uuid, "CurrentDisplayPrice": {"CurrencyCode": "USD", "TotalAmount": 0}, @@ -381,6 +386,7 @@ def get_metadata(book): "Title": book.title, "WorkId": book_uuid, } + metadata.update(get_author(book)) if get_series(book): if sys.version_info < (3, 0): @@ -399,7 +405,7 @@ def get_metadata(book): @kobo.route("/v1/library/tags", methods=["POST", "DELETE"]) -@login_required +@requires_kobo_auth # Creates a Shelf with the given items, and returns the shelf's uuid. def HandleTagCreate(): # catch delete requests, otherwise the are handeld in the book delete handler @@ -434,6 +440,7 @@ def HandleTagCreate(): @kobo.route("/v1/library/tags/", methods=["DELETE", "PUT"]) +@requires_kobo_auth def HandleTagUpdate(tag_id): shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.uuid == tag_id, ub.Shelf.user_id == current_user.id).one_or_none() @@ -488,7 +495,7 @@ def add_items_to_shelf(items, shelf): @kobo.route("/v1/library/tags//items", methods=["POST"]) -@login_required +@requires_kobo_auth def HandleTagAddItem(tag_id): items = None try: @@ -518,7 +525,7 @@ def HandleTagAddItem(tag_id): @kobo.route("/v1/library/tags//items/delete", methods=["POST"]) -@login_required +@requires_kobo_auth def HandleTagRemoveItem(tag_id): items = None try: @@ -627,7 +634,7 @@ def create_kobo_tag(shelf): @kobo.route("/v1/library//state", methods=["GET", "PUT"]) -@login_required +@requires_kobo_auth def HandleStateRequest(book_uuid): book = calibre_db.get_book_by_uuid(book_uuid) if not book or not book.data: @@ -801,7 +808,7 @@ def TopLevelEndpoint(): @kobo.route("/v1/library/", methods=["DELETE"]) -@login_required +@requires_kobo_auth def HandleBookDeletionRequest(book_uuid): log.info("Kobo book deletion request received for book %s" % book_uuid) book = calibre_db.get_book_by_uuid(book_uuid) diff --git a/cps/oauth_bb.py b/cps/oauth_bb.py index d98ec50a..3777d751 100644 --- a/cps/oauth_bb.py +++ b/cps/oauth_bb.py @@ -287,7 +287,7 @@ if ub.oauth_support: flash(_(u"Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error") except NoResultFound: log.warning("oauth %s for user %d not found", provider, current_user.id) - flash(_(u"Not Linked to %(oauth)s.", oauth=oauth_check[provider]), category="error") + flash(_(u"Not Linked to %(oauth)s", oauth=provider), category="error") return redirect(url_for('web.profile')) @@ -355,4 +355,4 @@ if ub.oauth_support: @oauth.route('/unlink/google', methods=["GET"]) @login_required def google_login_unlink(): - return unlink_oauth(oauthblueprints[1]['blueprint'].name) + return unlink_oauth(oauthblueprints[1]['id']) diff --git a/cps/reverseproxy.py b/cps/reverseproxy.py index 42b64050..3bcbd3b7 100644 --- a/cps/reverseproxy.py +++ b/cps/reverseproxy.py @@ -77,6 +77,7 @@ class ReverseProxied(object): servr = environ.get('HTTP_X_FORWARDED_HOST', '') if servr: environ['HTTP_HOST'] = servr + self.proxied = True return self.app(environ, start_response) @property diff --git a/cps/services/SyncToken.py b/cps/services/SyncToken.py index 1dd4f084..f6db960b 100644 --- a/cps/services/SyncToken.py +++ b/cps/services/SyncToken.py @@ -27,7 +27,10 @@ except ImportError: from urllib.parse import unquote from flask import json -from .. import logger as log +from .. import logger + + +log = logger.create() def b64encode_json(json_data): @@ -45,7 +48,8 @@ def to_epoch_timestamp(datetime_object): def get_datetime_from_json(json_object, field_name): try: return datetime.utcfromtimestamp(json_object[field_name]) - except KeyError: + except (KeyError, OSError, OverflowError): + # OSError is thrown on Windows if timestamp is <1970 or >2038 return datetime.min diff --git a/cps/services/simpleldap.py b/cps/services/simpleldap.py index 0933a933..336b0f2c 100644 --- a/cps/services/simpleldap.py +++ b/cps/services/simpleldap.py @@ -65,7 +65,10 @@ def init_app(app, config): app.config['LDAP_GROUP_OBJECT_FILTER'] = config.config_ldap_group_object_filter app.config['LDAP_GROUP_MEMBERS_FIELD'] = config.config_ldap_group_members_field - _ldap.init_app(app) + try: + _ldap.init_app(app) + except RuntimeError as e: + log.error(e) def get_object_details(user=None, group=None, query_filter=None, dn_only=False): diff --git a/cps/templates/book_edit.html b/cps/templates/book_edit.html index 3d1d629b..be6d4a67 100644 --- a/cps/templates/book_edit.html +++ b/cps/templates/book_edit.html @@ -130,19 +130,20 @@ {% endif %} - {% if c.datatype in ['text', 'series'] and not c.is_multiple %} + {% if c.datatype == 'text' %} 0 %} - value="{{ book['custom_column_' ~ c.id][0].value }}" - {% endif %}> + value="{% for column in book['custom_column_' ~ c.id] %}{{ column.value.strip() }}{% if not loop.last %}, {% endif %}{% endfor %}"{% endif %}> {% endif %} - {% if c.datatype in ['text', 'series'] and c.is_multiple %} + {% if c.datatype == 'series' %} 0 %} - value="{% for column in book['custom_column_' ~ c.id] %}{{ column.value.strip() }}{% if not loop.last %}, {% endif %}{% endfor %}"{% endif %}> + value="{% for column in book['custom_column_' ~ c.id] %} {{ '%s [%s]' % (book['custom_column_' ~ c.id][0].value, book['custom_column_' ~ c.id][0].extra|formatfloat(2)) }}{% if not loop.last %}, {% endif %}{% endfor %}" + {% endif %}> {% endif %} + {% if c.datatype == 'enumeration' %} 0 %} - value="{{ '%d' % (book['custom_column_' ~ c.id][0].value / 2) }}" + value="{{ '%.1f' % (book['custom_column_' ~ c.id][0].value / 2) }}" {% endif %}> {% endif %} diff --git a/cps/templates/config_edit.html b/cps/templates/config_edit.html index 77a60c1b..2d1ae560 100644 --- a/cps/templates/config_edit.html +++ b/cps/templates/config_edit.html @@ -30,20 +30,20 @@
{% if gdriveError %}
-
{% else %} {% if show_authenticate_google_drive and g.user.is_authenticated and config.config_use_google_drive %}
- {{_('Authenticate Google Drive')}} + {{_('Authenticate Google Drive')}}
{% else %} {% if show_authenticate_google_drive and g.user.is_authenticated and not config.config_use_google_drive %} -
{{_('Please hit submit to continue with setup')}}
+
{{_('Please hit save to continue with setup')}}
{% endif %} - {% if not g.user.is_authenticated %} + {% if not g.user.is_authenticated and show_login_button %}
{{_('Please finish Google Drive setup after login')}}
{% endif %} {% if g.user.is_authenticated %} diff --git a/cps/templates/detail.html b/cps/templates/detail.html index 1210a6ca..a0943122 100644 --- a/cps/templates/detail.html +++ b/cps/templates/detail.html @@ -174,7 +174,7 @@ {{ c.name }}: {% for column in entry['custom_column_' ~ c.id] %} {% if c.datatype == 'rating' %} - {{ '%d' % (column.value / 2) }} + {{ (column.value / 2)|formatfloat }} {% else %} {% if c.datatype == 'bool' %} {% if column.value == true %} @@ -182,9 +182,17 @@ {% else %} {% endif %} + {% else %} + {% if c.datatype == 'float' %} + {{ column.value|formatfloat(2) }} + {% else %} + {% if c.datatype == 'series' %} + {{ '%s [%s]' % (column.value, column.extra|formatfloat(2)) }} {% else %} {{ column.value }} {% endif %} + {% endif %} + {% endif %} {% endif %} {% endfor %} {% endif %} diff --git a/cps/templates/search_form.html b/cps/templates/search_form.html index 25e30d35..f1ea250f 100644 --- a/cps/templates/search_form.html +++ b/cps/templates/search_form.html @@ -165,7 +165,7 @@ {% endif %} {% if c.datatype == 'rating' %} - + {% endif %}
{% endfor %} diff --git a/cps/templates/stats.html b/cps/templates/stats.html index 69712cc4..966abf2a 100644 --- a/cps/templates/stats.html +++ b/cps/templates/stats.html @@ -25,6 +25,7 @@ +{% if g.user.role_admin() %}

{{_('Linked Libraries')}}

@@ -44,4 +45,5 @@ {% endfor %}
+{% endif %} {% endblock %} diff --git a/cps/translations/cs/LC_MESSAGES/messages.po b/cps/translations/cs/LC_MESSAGES/messages.po index 6ef881ed..3dca9fe8 100644 --- a/cps/translations/cs/LC_MESSAGES/messages.po +++ b/cps/translations/cs/LC_MESSAGES/messages.po @@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive" msgstr "Ověřit Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Klikněte na odeslat pro pokračování v nastavení" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/de/LC_MESSAGES/messages.po b/cps/translations/de/LC_MESSAGES/messages.po index a1372adb..4d9725e5 100644 --- a/cps/translations/de/LC_MESSAGES/messages.po +++ b/cps/translations/de/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "Google Drive authentifizieren" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Bitte auf Abschicken drücken, um mit dem Setup fortzufahren" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/es/LC_MESSAGES/messages.po b/cps/translations/es/LC_MESSAGES/messages.po index 99bac86e..4bd142ca 100644 --- a/cps/translations/es/LC_MESSAGES/messages.po +++ b/cps/translations/es/LC_MESSAGES/messages.po @@ -1560,7 +1560,7 @@ msgid "Authenticate Google Drive" msgstr "Autentificar Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Por favor, pulsa enviar para continuar con la configuración" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/fi/LC_MESSAGES/messages.po b/cps/translations/fi/LC_MESSAGES/messages.po index 09b133f1..26d644fb 100644 --- a/cps/translations/fi/LC_MESSAGES/messages.po +++ b/cps/translations/fi/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "Autentikoi Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Ole hyvä ja paina lähetä jatkaaksesi asennusta" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/fr/LC_MESSAGES/messages.po b/cps/translations/fr/LC_MESSAGES/messages.po index 6860cc52..5ba89d85 100644 --- a/cps/translations/fr/LC_MESSAGES/messages.po +++ b/cps/translations/fr/LC_MESSAGES/messages.po @@ -1571,7 +1571,7 @@ msgid "Authenticate Google Drive" msgstr "Authentification Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Veuillez cliquer sur soumettre pour continuer l’initialisation" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/hu/LC_MESSAGES/messages.po b/cps/translations/hu/LC_MESSAGES/messages.po index 0b0f71b4..4550161a 100644 --- a/cps/translations/hu/LC_MESSAGES/messages.po +++ b/cps/translations/hu/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "Google Drive hitelesítés" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "A beállítás folytatásához kattints a Küldés gombra" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/it/LC_MESSAGES/messages.po b/cps/translations/it/LC_MESSAGES/messages.po index 8c544937..1e069ae6 100644 --- a/cps/translations/it/LC_MESSAGES/messages.po +++ b/cps/translations/it/LC_MESSAGES/messages.po @@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive" msgstr "Autenticazione Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Per favore premi invio per proseguire con la configurazione" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/ja/LC_MESSAGES/messages.po b/cps/translations/ja/LC_MESSAGES/messages.po index 4ae537dc..01b8e37f 100644 --- a/cps/translations/ja/LC_MESSAGES/messages.po +++ b/cps/translations/ja/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "Googleドライブを認証" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "決定を押して設定を続けてください" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/km/LC_MESSAGES/messages.po b/cps/translations/km/LC_MESSAGES/messages.po index 10d2d6af..e8def12b 100644 --- a/cps/translations/km/LC_MESSAGES/messages.po +++ b/cps/translations/km/LC_MESSAGES/messages.po @@ -1558,7 +1558,7 @@ msgid "Authenticate Google Drive" msgstr "វាយបញ្ចូលគណនី Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/nl/LC_MESSAGES/messages.po b/cps/translations/nl/LC_MESSAGES/messages.po index 4ce11eb2..eda57375 100644 --- a/cps/translations/nl/LC_MESSAGES/messages.po +++ b/cps/translations/nl/LC_MESSAGES/messages.po @@ -1558,7 +1558,7 @@ msgid "Authenticate Google Drive" msgstr "Google Drive goedkeuren" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Druk op 'Opslaan' om door te gaan met instellen" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/pl/LC_MESSAGES/messages.po b/cps/translations/pl/LC_MESSAGES/messages.po index e6dc6b45..8718b9b6 100644 --- a/cps/translations/pl/LC_MESSAGES/messages.po +++ b/cps/translations/pl/LC_MESSAGES/messages.po @@ -1570,7 +1570,7 @@ msgid "Authenticate Google Drive" msgstr "Uwierzytelnij Dysk Google" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Kliknij przycisk, aby kontynuować instalację" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/ru/LC_MESSAGES/messages.po b/cps/translations/ru/LC_MESSAGES/messages.po index 8b265dab..403e0c94 100644 --- a/cps/translations/ru/LC_MESSAGES/messages.po +++ b/cps/translations/ru/LC_MESSAGES/messages.po @@ -1558,7 +1558,7 @@ msgid "Authenticate Google Drive" msgstr "Аутентификация Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Пожалуйста, нажмите «Отправить», чтобы продолжить настройку" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/sv/LC_MESSAGES/messages.po b/cps/translations/sv/LC_MESSAGES/messages.po index b41fd929..ceb839d4 100644 --- a/cps/translations/sv/LC_MESSAGES/messages.po +++ b/cps/translations/sv/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "Autentisera Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Klicka på skicka för att fortsätta med installationen" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/tr/LC_MESSAGES/messages.po b/cps/translations/tr/LC_MESSAGES/messages.po index b6ee8906..4abd7bc2 100644 --- a/cps/translations/tr/LC_MESSAGES/messages.po +++ b/cps/translations/tr/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "Google Drive Doğrula" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "Kuruluma devam etmek için Gönder'e tıklayın" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/uk/LC_MESSAGES/messages.po b/cps/translations/uk/LC_MESSAGES/messages.po index 41c448b2..d7dc9af3 100644 --- a/cps/translations/uk/LC_MESSAGES/messages.po +++ b/cps/translations/uk/LC_MESSAGES/messages.po @@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive" msgstr "Автентифікація Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "" #: cps/templates/config_edit.html:47 diff --git a/cps/translations/zh_Hans_CN/LC_MESSAGES/messages.po b/cps/translations/zh_Hans_CN/LC_MESSAGES/messages.po index 76b99287..759263c8 100644 --- a/cps/translations/zh_Hans_CN/LC_MESSAGES/messages.po +++ b/cps/translations/zh_Hans_CN/LC_MESSAGES/messages.po @@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive" msgstr "认证 Google Drive" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "请点击提交以继续设置" #: cps/templates/config_edit.html:47 diff --git a/cps/ub.py b/cps/ub.py index 1aced154..9a80320e 100644 --- a/cps/ub.py +++ b/cps/ub.py @@ -177,20 +177,20 @@ class UserBase: return self.check_visibility(constants.DETAIL_RANDOM) def list_denied_tags(self): - mct = self.denied_tags.split(",") - return [t.strip() for t in mct] + mct = self.denied_tags or "" + return [t.strip() for t in mct.split(",")] def list_allowed_tags(self): - mct = self.allowed_tags.split(",") - return [t.strip() for t in mct] + mct = self.allowed_tags or "" + return [t.strip() for t in mct.split(",")] def list_denied_column_values(self): - mct = self.denied_column_value.split(",") - return [t.strip() for t in mct] + mct = self.denied_column_value or "" + return [t.strip() for t in mct.split(",")] def list_allowed_column_values(self): - mct = self.allowed_column_value.split(",") - return [t.strip() for t in mct] + mct = self.allowed_column_value or "" + return [t.strip() for t in mct.split(",")] def __repr__(self): return '' % self.nickname @@ -478,34 +478,34 @@ def migrate_Database(session): ArchivedBook.__table__.create(bind=engine) if not engine.dialect.has_table(engine.connect(), "registration"): ReadBook.__table__.create(bind=engine) - conn = engine.connect() - conn.execute("insert into registration (domain, allow) values('%.%',1)") + with engine.connect() as conn: + conn.execute("insert into registration (domain, allow) values('%.%',1)") session.commit() try: session.query(exists().where(Registration.allow)).scalar() session.commit() except exc.OperationalError: # Database is not compatible, some columns are missing - conn = engine.connect() - conn.execute("ALTER TABLE registration ADD column 'allow' INTEGER") - conn.execute("update registration set 'allow' = 1") + with engine.connect() as conn: + conn.execute("ALTER TABLE registration ADD column 'allow' INTEGER") + conn.execute("update registration set 'allow' = 1") session.commit() try: session.query(exists().where(RemoteAuthToken.token_type)).scalar() session.commit() except exc.OperationalError: # Database is not compatible, some columns are missing - conn = engine.connect() - conn.execute("ALTER TABLE remote_auth_token ADD column 'token_type' INTEGER DEFAULT 0") - conn.execute("update remote_auth_token set 'token_type' = 0") + with engine.connect() as conn: + conn.execute("ALTER TABLE remote_auth_token ADD column 'token_type' INTEGER DEFAULT 0") + conn.execute("update remote_auth_token set 'token_type' = 0") session.commit() try: session.query(exists().where(ReadBook.read_status)).scalar() except exc.OperationalError: - conn = engine.connect() - conn.execute("ALTER TABLE book_read_link ADD column 'read_status' INTEGER DEFAULT 0") - conn.execute("UPDATE book_read_link SET 'read_status' = 1 WHERE is_read") - conn.execute("ALTER TABLE book_read_link ADD column 'last_modified' DATETIME") - conn.execute("ALTER TABLE book_read_link ADD column 'last_time_started_reading' DATETIME") - conn.execute("ALTER TABLE book_read_link ADD column 'times_started_reading' INTEGER DEFAULT 0") + with engine.connect() as conn: + conn.execute("ALTER TABLE book_read_link ADD column 'read_status' INTEGER DEFAULT 0") + conn.execute("UPDATE book_read_link SET 'read_status' = 1 WHERE is_read") + conn.execute("ALTER TABLE book_read_link ADD column 'last_modified' DATETIME") + conn.execute("ALTER TABLE book_read_link ADD column 'last_time_started_reading' DATETIME") + conn.execute("ALTER TABLE book_read_link ADD column 'times_started_reading' INTEGER DEFAULT 0") session.commit() test = session.query(ReadBook).filter(ReadBook.last_modified == None).all() for book in test: @@ -514,11 +514,11 @@ def migrate_Database(session): try: session.query(exists().where(Shelf.uuid)).scalar() except exc.OperationalError: - conn = engine.connect() - conn.execute("ALTER TABLE shelf ADD column 'uuid' STRING") - conn.execute("ALTER TABLE shelf ADD column 'created' DATETIME") - conn.execute("ALTER TABLE shelf ADD column 'last_modified' DATETIME") - conn.execute("ALTER TABLE book_shelf_link ADD column 'date_added' DATETIME") + with engine.connect() as conn: + conn.execute("ALTER TABLE shelf ADD column 'uuid' STRING") + conn.execute("ALTER TABLE shelf ADD column 'created' DATETIME") + conn.execute("ALTER TABLE shelf ADD column 'last_modified' DATETIME") + conn.execute("ALTER TABLE book_shelf_link ADD column 'date_added' DATETIME") for shelf in session.query(Shelf).all(): shelf.uuid = str(uuid.uuid4()) shelf.created = datetime.datetime.now() @@ -529,31 +529,31 @@ def migrate_Database(session): # Handle table exists, but no content cnt = session.query(Registration).count() if not cnt: - conn = engine.connect() - conn.execute("insert into registration (domain, allow) values('%.%',1)") + with engine.connect() as conn: + conn.execute("insert into registration (domain, allow) values('%.%',1)") session.commit() try: session.query(exists().where(BookShelf.order)).scalar() except exc.OperationalError: # Database is not compatible, some columns are missing - conn = engine.connect() - conn.execute("ALTER TABLE book_shelf_link ADD column 'order' INTEGER DEFAULT 1") + with engine.connect() as conn: + conn.execute("ALTER TABLE book_shelf_link ADD column 'order' INTEGER DEFAULT 1") session.commit() try: create = False session.query(exists().where(User.sidebar_view)).scalar() except exc.OperationalError: # Database is not compatible, some columns are missing - conn = engine.connect() - conn.execute("ALTER TABLE user ADD column `sidebar_view` Integer DEFAULT 1") + with engine.connect() as conn: + conn.execute("ALTER TABLE user ADD column `sidebar_view` Integer DEFAULT 1") session.commit() create = True try: if create: - conn = engine.connect() - conn.execute("SELECT language_books FROM user") + with engine.connect() as conn: + conn.execute("SELECT language_books FROM user") session.commit() except exc.OperationalError: - conn = engine.connect() - conn.execute("UPDATE user SET 'sidebar_view' = (random_books* :side_random + language_books * :side_lang " + with engine.connect() as conn: + conn.execute("UPDATE user SET 'sidebar_view' = (random_books* :side_random + language_books * :side_lang " "+ series_books * :side_series + category_books * :side_category + hot_books * " ":side_hot + :side_autor + :detail_random)", {'side_random': constants.SIDEBAR_RANDOM, 'side_lang': constants.SIDEBAR_LANGUAGE, @@ -564,35 +564,29 @@ def migrate_Database(session): try: session.query(exists().where(User.denied_tags)).scalar() except exc.OperationalError: # Database is not compatible, some columns are missing - conn = engine.connect() - conn.execute("ALTER TABLE user ADD column `denied_tags` String DEFAULT ''") - conn.execute("ALTER TABLE user ADD column `allowed_tags` String DEFAULT ''") - conn.execute("ALTER TABLE user ADD column `denied_column_value` DEFAULT ''") - conn.execute("ALTER TABLE user ADD column `allowed_column_value` DEFAULT ''") + with engine.connect() as conn: + conn.execute("ALTER TABLE user ADD column `denied_tags` String DEFAULT ''") + conn.execute("ALTER TABLE user ADD column `allowed_tags` String DEFAULT ''") + conn.execute("ALTER TABLE user ADD column `denied_column_value` String DEFAULT ''") + conn.execute("ALTER TABLE user ADD column `allowed_column_value` String DEFAULT ''") session.commit() - #try: - # session.query(exists().where(User.series_view)).scalar() - #except exc.OperationalError: - # conn = engine.connect() - # conn.execute("ALTER TABLE user ADD column `series_view` VARCHAR(10) DEFAULT 'list'") try: session.query(exists().where(User.view_settings)).scalar() except exc.OperationalError: - conn = engine.connect() - conn.execute("ALTER TABLE user ADD column `view_settings` JSON default '{}'") - session.commit() + with engine.connect() as conn: + conn.execute("ALTER TABLE user ADD column `series_view` VARCHAR(10) DEFAULT 'list'") if session.query(User).filter(User.role.op('&')(constants.ROLE_ANONYMOUS) == constants.ROLE_ANONYMOUS).first() \ is None: create_anonymous_user(session) try: # check if one table with autoincrement is existing (should be user table) - conn = engine.connect() - conn.execute("SELECT COUNT(*) FROM sqlite_sequence WHERE name='user'") + with engine.connect() as conn: + conn.execute("SELECT COUNT(*) FROM sqlite_sequence WHERE name='user'") except exc.OperationalError: # Create new table user_id and copy contents of table user into it - conn = engine.connect() - conn.execute("CREATE TABLE user_id (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT," + with engine.connect() as conn: + conn.execute("CREATE TABLE user_id (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT," "nickname VARCHAR(64)," "email VARCHAR(120)," "role SMALLINT," @@ -605,20 +599,19 @@ def migrate_Database(session): "view_settings VARCHAR," "UNIQUE (nickname)," "UNIQUE (email))") - conn.execute("INSERT INTO user_id(id, nickname, email, role, password, kindle_mail,locale," - # "sidebar_view, default_language, series_view) " - "sidebar_view, default_language) " + conn.execute("INSERT INTO user_id(id, nickname, email, role, password, kindle_mail,locale," + "sidebar_view, default_language, series_view) " "SELECT id, nickname, email, role, password, kindle_mail, locale," "sidebar_view, default_language FROM user") - # delete old user table and rename new user_id table to user: - conn.execute("DROP TABLE user") - conn.execute("ALTER TABLE user_id RENAME TO user") + # delete old user table and rename new user_id table to user: + conn.execute("DROP TABLE user") + conn.execute("ALTER TABLE user_id RENAME TO user") session.commit() # Remove login capability of user Guest try: - conn = engine.connect() - conn.execute("UPDATE user SET password='' where nickname = 'Guest' and password !=''") + with engine.connect() as conn: + conn.execute("UPDATE user SET password='' where nickname = 'Guest' and password !=''") session.commit() except exc.OperationalError: print('Settings database is not writeable. Exiting...') @@ -686,8 +679,6 @@ def init_db(app_db_path): app_DB_path = app_db_path engine = create_engine(u'sqlite:///{0}'.format(app_db_path), echo=False) - # engine.execute("attach database '{0}' as app_settings;".format(app_db_path)) - Session = sessionmaker() Session.configure(bind=engine) diff --git a/cps/uploader.py b/cps/uploader.py index 1323e3d0..e693d8e8 100644 --- a/cps/uploader.py +++ b/cps/uploader.py @@ -35,7 +35,7 @@ except ImportError: lxmlversion = None try: - from wand.image import Image + from wand.image import Image, Color from wand import version as ImageVersion from wand.exceptions import PolicyError use_generic_pdf_cover = False @@ -116,8 +116,8 @@ def default_meta(tmp_file_path, original_file_name, original_file_extension): def pdf_meta(tmp_file_path, original_file_name, original_file_extension): doc_info = None if use_pdf_meta: - doc_info = PdfFileReader(open(tmp_file_path, 'rb')).getDocumentInfo() - + with open(tmp_file_path, 'rb') as f: + doc_info = PdfFileReader(f).getDocumentInfo() if doc_info: author = doc_info.author if doc_info.author else u'Unknown' title = doc_info.title if doc_info.title else original_file_name @@ -149,6 +149,9 @@ def pdf_preview(tmp_file_path, tmp_dir): img.options["pdf:use-cropbox"] = "true" img.read(filename=tmp_file_path + '[0]', resolution=150) img.compression_quality = 88 + if img.alpha_channel: + img.alpha_channel = 'remove' + img.background_color = Color('white') img.save(filename=os.path.join(tmp_dir, cover_file_name)) return cover_file_name except PolicyError as ex: @@ -156,6 +159,7 @@ def pdf_preview(tmp_file_path, tmp_dir): return None except Exception as ex: log.warning('Cannot extract cover image, using default: %s', ex) + log.warning('On Windows this error could be caused by missing ghostscript') return None diff --git a/cps/web.py b/cps/web.py index dcd34571..87c48a20 100644 --- a/cps/web.py +++ b/cps/web.py @@ -1316,7 +1316,7 @@ def advanced_search(): db.cc_classes[c.id].value == custom_query)) elif c.datatype == 'rating': q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any( - db.cc_classes[c.id].value == int(custom_query) * 2)) + db.cc_classes[c.id].value == int(float(custom_query) * 2))) else: q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any( func.lower(db.cc_classes[c.id].value).ilike("%" + custom_query + "%"))) @@ -1349,6 +1349,9 @@ def advanced_search_form(): def get_cover(book_id): return get_book_cover(book_id) +@web.route("/robots.txt") +def get_robots(): + return send_from_directory(constants.STATIC_DIR, "robots.txt") @web.route("/show//", defaults={'anyname': 'None'}) @web.route("/show///") diff --git a/messages.pot b/messages.pot index 8b630b6c..078f1669 100644 --- a/messages.pot +++ b/messages.pot @@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive" msgstr "" #: cps/templates/config_edit.html:44 -msgid "Please hit submit to continue with setup" +msgid "Please hit save to continue with setup" msgstr "" #: cps/templates/config_edit.html:47 diff --git a/test/Calibre-Web TestSummary.html b/test/Calibre-Web TestSummary.html index 43913dbe..70269acb 100755 --- a/test/Calibre-Web TestSummary.html +++ b/test/Calibre-Web TestSummary.html @@ -36,17 +36,17 @@
-

Start Time: 2020-06-28 20:44:31

+

Start Time: 2020-08-14 19:46:42

-

Stop Time: 2020-06-28 21:48:11

+

Stop Time: 2020-08-14 21:02:08

-

Duration: 53:53 min

+

Duration: 1h 243 min

@@ -570,8 +570,8 @@ AssertionError: False is not true test_edit_books.TestEditBooks 33 - 30 - 1 + 29 + 2 0 2 @@ -894,11 +894,33 @@ AssertionError: False is not true - +
test_upload_book_pdf
- PASS + +
+ FAIL +
+ + + + @@ -1070,13 +1092,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye test_helper.CalibreHelper - 13 - 13 + 15 + 15 0 0 0 - Detail + Detail @@ -1198,6 +1220,24 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye + + + +
test_random_password
+ + PASS + + + + + + +
test_whitespaces
+ + PASS + + + @@ -1289,13 +1329,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye test_ldap.test_ldap_login - 9 - 9 + 10 + 10 0 0 0 - Detail + Detail @@ -1374,6 +1414,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye + +
test_ldap_kobo_sync
+ + PASS + + + + +
test_ldap_opds_download_book
@@ -1480,13 +1529,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye test_login.test_Login - 10 - 10 + 11 + 11 0 0 0 - Detail + Detail @@ -1581,6 +1630,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye + + + +
test_robots
+ + PASS + + + @@ -1813,13 +1871,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye test_register.test_register - 6 - 6 + 7 + 7 0 0 0 - Detail + Detail @@ -1854,7 +1912,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye -
test_registering_user
+
test_registering_only_email
PASS @@ -1863,7 +1921,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye -
test_registering_user_fail
+
test_registering_user
PASS @@ -1871,6 +1929,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye + +
test_registering_user_fail
+ + PASS + + + + +
test_user_change_password
@@ -2578,9 +2645,9 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye Total - 223 - 215 - 2 + 228 + 219 + 3 0 6   @@ -2610,13 +2677,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye Platform - Linux 5.3.0-59-generic #53~18.04.1-Ubuntu SMP Thu Jun 4 14:58:26 UTC 2020 x86_64 x86_64 + Linux 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 Basic Python - 3.7.5 + 3.8.2 Basic @@ -2730,7 +2797,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye Pillow - 7.1.2 + 7.2.0 testCoverEditBooks @@ -2742,31 +2809,31 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye lxml - 4.5.1 + 4.5.2 TestEditAdditionalBooks Pillow - 7.1.2 + 7.2.0 TestEditAdditionalBooks rarfile - 3.1 + 4.0 TestEditAdditionalBooks lxml - 4.5.1 + 4.5.2 TestEditBooks Pillow - 7.1.2 + 7.2.0 TestEditBooks @@ -2788,9 +2855,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye test_ldap_login + + jsonschema + 3.2.0 + test_ldap_login + + python-ldap - 3.3.0 + 3.3.1 test_ldap_login @@ -2802,7 +2875,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye SQLAlchemy-Utils - 0.36.6 + 0.36.8 test_OAuth_login @@ -2814,7 +2887,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye