Merge with master

4 years ago · ad144922fb
parent f8139f3198 b9c0f2a3d3
commit ad144922fb
40 changed files with 370 additions and 248 deletions
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -1,49 +1,46 @@
 ## How to contribute to Calibre-Web

-First of all, we would like to thank you for reading this text. we are happy you are willing to contribute to Calibre-Web
+First of all, we would like to thank you for reading this text. We are happy you are willing to contribute to Calibre-Web.

 ### **General**

-**Communication language** is english. Google translated texts are not as bad as you might think, they are usually understandable, so don't worry if you generate your post that way.
+**Communication language** is English. Google translated texts are not as bad as you might think, they are usually understandable, so don't worry if you generate your post that way.

 **Calibre-Web** is not **Calibre**. If you are having a question regarding Calibre please post this at their [repository](https://github.com/kovidgoyal/calibre).

-**Docker-Containers** of Calibre-Web are maintained by different persons than the people who drive Calibre-Web. If we come to the conclusion during our analysis that the problem is related to Docker, we might ask you to open a new issue at the reprository of the Docker Container.
+**Docker-Containers** of Calibre-Web are maintained by different persons than the people who drive Calibre-Web. If we come to the conclusion during our analysis that the problem is related to Docker, we might ask you to open a new issue at the repository of the Docker Container.

-If you are having **Basic Installation Problems** with python or it's dependencies, please consider using your favorite search engine to find a solution. In case you can't find a solution, we are happy to help you.
+If you are having **Basic Installation Problems** with python or its dependencies, please consider using your favorite search engine to find a solution. In case you can't find a solution, we are happy to help you.

-We can offer only very limited support regarding configuration of **Reverse-Proxy Installations**, **OPDS-Reader** or other programs in combination with Calibre-Web. 
+We can offer only very limited support regarding configuration of **Reverse-Proxy Installations**, **OPDS-Reader** or other programs in combination with Calibre-Web.

 ### **Translation**

-Some of the user languages in Calibre-Web having missing translations. We are happy to add the missing texts if you translate them. Create a Pull Request, create an issue with the .po file attached, or write an email to "ozzie.fernandez.isaacs@googlemail.com" with attached translation file. To display all book languages in your native language an additional file is used (iso_language_names.py). The content of this file is autogenerated with the corresponding translations of Calibre, please do not edit this file on your own. 
+Some of the user languages in Calibre-Web having missing translations. We are happy to add the missing texts if you translate them. Create a Pull Request, create an issue with the .po file attached, or write an email to "ozzie.fernandez.isaacs@googlemail.com" with attached translation file. To display all book languages in your native language an additional file is used (iso_language_names.py). The content of this file is auto-generated with the corresponding translations of Calibre, please do not edit this file on your own.

 ### **Documentation**

-The Calibre-Web documentation is hosted in the Github [Wiki](https://github.com/janeczku/calibre-web/wiki). The Wiki is open to everybody, if you find a problem, feel free to correct it. If information is missing, you are welcome to add it. The content will be reviewed time by time. Please try to be consitent with the form with the other Wiki pages (e.g. the project name is Calibre-Web with 2 capital letters and a dash in between).
+The Calibre-Web documentation is hosted in the Github [Wiki](https://github.com/janeczku/calibre-web/wiki). The Wiki is open to everybody, if you find a problem, feel free to correct it. If information is missing, you are welcome to add it. The content will be reviewed time by time. Please try to be consistent with the form with the other Wiki pages (e.g. the project name is Calibre-Web with 2 capital letters and a dash in between).

 ### **Reporting a bug**

-Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Please write intead an email to "ozzie.fernandez.isaacs@googlemail.com".
+Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Instead, please write an email to "ozzie.fernandez.isaacs@googlemail.com".

 Ensure the ***bug was not already reported** by searching on GitHub under [Issues](https://github.com/janeczku/calibre-web/issues). Please also check if a solution for your problem can be found in the [wiki](https://github.com/janeczku/calibre-web/wiki).

-If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=bug_report.md&title=). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue. 
+If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=bug_report.md&title=). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue.

 ### **Feature Request**

-If there is a feature missing in Calibre-Web and you can't find a feature request in the [Issues](https://github.com/janeczku/calibre-web/issues) section, you could create a [feature request](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=feature_request.md&title=). 
-We will not extend Calibre-Web with any more login abilitys or add further files storages, or file syncing ability. Furthermore Calibre-Web is made for home usage for company inhouse usage, so requests regarding any sorts of social interaction capability, payment routines, search engine or web site analytics integration will not be implemeted. 
+If there is a feature missing in Calibre-Web and you can't find a feature request in the [Issues](https://github.com/janeczku/calibre-web/issues) section, you could create a [feature request](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=feature_request.md&title=).
+We will not extend Calibre-Web with any more login abilities or add further files storages, or file syncing ability. Furthermore Calibre-Web is made for home usage for company in-house usage, so requests regarding any sorts of social interaction capability, payment routines, search engine or web site analytics integration will not be implemented.

 ### **Contributing code to Calibre-Web**

 Open a new GitHub pull request with the patch. Ensure the PR description clearly describes the problem and solution. Include the relevant issue number if applicable.

-In case your code enhances features of Calibre-Web: Create your pull request for the development branch if your enhancement consits of more than some lines of code in a local section of Calibre-Webs code. This makes it easier to test it and check all implication before it's made public. 
-
-Please check if your code runs on Python 2.7 (still necessary in 2020) and mainly on python 3. If possible and the feature is related to operating system functions, try to check it on Windows and Linux. 
-Calibre-Web is automatically tested on Linux in combination with python 3.7. The code for testing is in a [seperate repo](https://github.com/OzzieIsaacs/calibre-web-test) on Github. It uses unittest and performs real system tests with selenium, would be great if you could consider also writing some tests. 
-A static code analysis is done by Codacy, but it's partly broken and doesn't run automatically. You could check your code with ESLint before contributing, a configuration file can be found in the projects root folder. 
-
-
+In case your code enhances features of Calibre-Web: Create your pull request for the development branch if your enhancement consists of more than some lines of code in a local section of Calibre-Webs code. This makes it easier to test it and check all implication before it's made public.

+Please check if your code runs on Python 2.7 (still necessary in 2020) and mainly on python 3. If possible and the feature is related to operating system functions, try to check it on Windows and Linux.
+Calibre-Web is automatically tested on Linux in combination with python 3.7. The code for testing is in a [separate repo](https://github.com/OzzieIsaacs/calibre-web-test) on Github. It uses unit tests and performs real system tests with selenium; it would be great if you could consider also writing some tests.
+A static code analysis is done by Codacy, but it's partly broken and doesn't run automatically. You could check your code with ESLint before contributing, a configuration file can be found in the projects root folder.
--- a/cps/admin.py
+++ b/cps/admin.py
@ -99,7 +99,7 @@ def shutdown():

    if task == 2:
        log.warning("reconnecting to calibre database")
-        calibre_db.setup_db(config, ub.app_DB_path)
+        calibre_db.reconnect_db(config, ub.app_DB_path)
        showtext['text'] = _(u'Reconnect successful')
        return json.dumps(showtext)

--- a/cps/config_sql.py
+++ b/cps/config_sql.py
@ -22,7 +22,7 @@ import os
 import json
 import sys

-from sqlalchemy import exc, Column, String, Integer, SmallInteger, Boolean, BLOB
+from sqlalchemy import exc, Column, String, Integer, SmallInteger, Boolean, BLOB, JSON
 from sqlalchemy.ext.declarative import declarative_base

 from . import constants, cli, logger, ub
@ -92,7 +92,7 @@ class _Settings(_Base):

    config_use_google_drive = Column(Boolean, default=False)
    config_google_drive_folder = Column(String)
-    config_google_drive_watch_changes_response = Column(String)
+    config_google_drive_watch_changes_response = Column(JSON, default={})

    config_use_goodreads = Column(Boolean, default=False)
    config_goodreads_api_key = Column(String)
@ -102,7 +102,6 @@ class _Settings(_Base):

    config_kobo_proxy = Column(Boolean, default=False)

-
    config_ldap_provider_url = Column(String, default='example.org')
    config_ldap_port = Column(SmallInteger, default=389)
    config_ldap_authentication = Column(SmallInteger, default=constants.LDAP_AUTH_SIMPLE)
@ -215,20 +214,20 @@ class _ConfigSQL(object):
        return self.show_element_new_user(constants.DETAIL_RANDOM)

    def list_denied_tags(self):
-        mct = self.config_denied_tags.split(",")
-        return [t.strip() for t in mct]
+        mct = self.config_denied_tags or ""
+        return [t.strip() for t in mct.split(",")]

    def list_allowed_tags(self):
-        mct = self.config_allowed_tags.split(",")
-        return [t.strip() for t in mct]
+        mct = self.config_allowed_tags or ""
+        return [t.strip() for t in mct.split(",")]

    def list_denied_column_values(self):
-        mct = self.config_denied_column_value.split(",")
-        return [t.strip() for t in mct]
+        mct = self.config_denied_column_value or ""
+        return [t.strip() for t in mct.split(",")]

    def list_allowed_column_values(self):
-        mct = self.config_allowed_column_value.split(",")
-        return [t.strip() for t in mct]
+        mct = self.config_allowed_column_value or ""
+        return [t.strip() for t in mct.split(",")]

    def get_log_level(self):
        return logger.get_level_name(self.config_log_level)
@ -281,10 +280,6 @@ class _ConfigSQL(object):
                        v = column.default.arg
                setattr(self, k, v)

-        if self.config_google_drive_watch_changes_response:
-            self.config_google_drive_watch_changes_response = \
-                json.loads(self.config_google_drive_watch_changes_response)
-
        have_metadata_db = bool(self.config_calibre_dir)
        if have_metadata_db:
            if not self.config_use_google_drive:
@ -303,10 +298,6 @@ class _ConfigSQL(object):
        '''Apply all configuration values to the underlying storage.'''
        s = self._read_from_storage()  # type: _Settings

-        if self.config_google_drive_watch_changes_response:
-            self.config_google_drive_watch_changes_response = json.dumps(
-                self.config_google_drive_watch_changes_response)
-
        for k, v in self.__dict__.items():
            if k[0] == '_':
                continue
@ -361,10 +352,10 @@ def _migrate_table(session, orm_class):

 def autodetect_calibre_binary():
    if sys.platform == "win32":
-        calibre_path = ["C:\\program files\calibre\ebook-convert.exe",
-                        "C:\\program files(x86)\calibre\ebook-convert.exe",
-                        "C:\\program files(x86)\calibre2\ebook-convert.exe",
-                        "C:\\program files\calibre2\ebook-convert.exe"]
+        calibre_path = ["C:\\program files\\calibre\\ebook-convert.exe",
+                        "C:\\program files(x86)\\calibre\\ebook-convert.exe",
+                        "C:\\program files(x86)\\calibre2\\ebook-convert.exe",
+                        "C:\\program files\\calibre2\\ebook-convert.exe"]
    else:
        calibre_path = ["/opt/calibre/ebook-convert"]
    for element in calibre_path:
--- a/cps/db.py
+++ b/cps/db.py
@ -33,8 +33,10 @@ from sqlalchemy.orm import relationship, sessionmaker, scoped_session
 from sqlalchemy.orm.collections import InstrumentedList
 from sqlalchemy.ext.declarative import declarative_base, DeclarativeMeta
 from sqlalchemy.exc import OperationalError
+from sqlalchemy.pool import StaticPool
 from flask_login import current_user
 from sqlalchemy.sql.expression import and_, true, false, text, func, or_
+from sqlalchemy.ext.associationproxy import association_proxy
 from babel import Locale as LC
 from babel.core import UnknownLocaleError
 from flask_babel import gettext as _
@ -312,7 +314,7 @@ class Books(Base):
    flags = Column(Integer, nullable=False, default=1)

    authors = relationship('Authors', secondary=books_authors_link, backref='books')
-    tags = relationship('Tags', secondary=books_tags_link, backref='books',order_by="Tags.name")
+    tags = relationship('Tags', secondary=books_tags_link, backref='books', order_by="Tags.name")
    comments = relationship('Comments', backref='books')
    data = relationship('Data', backref='books')
    series = relationship('Series', secondary=books_series_link, backref='books')
@ -438,7 +440,6 @@ class CalibreDB(threading.Thread):
    def setup_db(self, config, app_db_path):
        self.config = config
        self.dispose()
-        # global engine

        if not config.config_calibre_dir:
            config.invalidate()
@ -450,11 +451,11 @@ class CalibreDB(threading.Thread):
            return False

        try:
-            #engine = create_engine('sqlite:///{0}'.format(dbpath),
            self.engine = create_engine('sqlite://',
                                   echo=False,
                                   isolation_level="SERIALIZABLE",
-                                   connect_args={'check_same_thread': False})
+                                   connect_args={'check_same_thread': False},
+                                   poolclass=StaticPool)
            self.engine.execute("attach database '{}' as calibre;".format(dbpath))
            self.engine.execute("attach database '{}' as app_settings;".format(app_db_path))

@ -474,34 +475,46 @@ class CalibreDB(threading.Thread):
            books_custom_column_links = {}
            for row in cc:
                if row.datatype not in cc_exceptions:
-                    books_custom_column_links[row.id] = Table('books_custom_column_' + str(row.id) + '_link', Base.metadata,
-                                                              Column('book', Integer, ForeignKey('books.id'),
-                                                                     primary_key=True),
-                                                              Column('value', Integer,
-                                                                     ForeignKey('custom_column_' + str(row.id) + '.id'),
-                                                                     primary_key=True)
-                                                              )
+                    if row.datatype == 'series':
+                        dicttable = {'__tablename__': 'books_custom_column_' + str(row.id) + '_link',
+                                     'id': Column(Integer, primary_key=True),
+                                     'book': Column(Integer, ForeignKey('books.id'),
+                                                    primary_key=True),
+                                     'map_value': Column('value', Integer,
+                                                     ForeignKey('custom_column_' +
+                                                                str(row.id) + '.id'),
+                                                     primary_key=True),
+                                     'extra': Column(Float),
+                                     'asoc' : relationship('custom_column_' + str(row.id), uselist=False),
+                                     'value' : association_proxy('asoc', 'value')
+                                     }
+                        books_custom_column_links[row.id] = type(str('books_custom_column_' + str(row.id) + '_link'),
+                                                                 (Base,), dicttable)
+                    else:
+                        books_custom_column_links[row.id] = Table('books_custom_column_' + str(row.id) + '_link',
+                                                                  Base.metadata,
+                                                                  Column('book', Integer, ForeignKey('books.id'),
+                                                                         primary_key=True),
+                                                                  Column('value', Integer,
+                                                                         ForeignKey('custom_column_' +
+                                                                                    str(row.id) + '.id'),
+                                                                         primary_key=True)
+                                                                  )
                    cc_ids.append([row.id, row.datatype])
-                    if row.datatype == 'bool':
-                        ccdict = {'__tablename__': 'custom_column_' + str(row.id),
-                                  'id': Column(Integer, primary_key=True),
-                                  'book': Column(Integer, ForeignKey('books.id')),
-                                  'value': Column(Boolean)}
+
+                    ccdict = {'__tablename__': 'custom_column_' + str(row.id),
+                              'id': Column(Integer, primary_key=True)}
+                    if row.datatype == 'float':
+                        ccdict['value'] = Column(Float)
                    elif row.datatype == 'int':
-                        ccdict = {'__tablename__': 'custom_column_' + str(row.id),
-                                  'id': Column(Integer, primary_key=True),
-                                  'book': Column(Integer, ForeignKey('books.id')),
-                                  'value': Column(Integer)}
-                    elif row.datatype == 'float':
-                        ccdict = {'__tablename__': 'custom_column_' + str(row.id),
-                                  'id': Column(Integer, primary_key=True),
-                                  'book': Column(Integer, ForeignKey('books.id')),
-                                  'value': Column(Float)}
+                        ccdict['value'] = Column(Integer)
+                    elif row.datatype == 'bool':
+                        ccdict['value'] = Column(Boolean)
                    else:
-                        ccdict = {'__tablename__': 'custom_column_' + str(row.id),
-                                  'id': Column(Integer, primary_key=True),
-                                  'value': Column(String)}
-                    cc_classes[row.id] = type(str('Custom_Column_' + str(row.id)), (Base,), ccdict)
+                        ccdict['value'] = Column(String)
+                    if row.datatype in ['float', 'int', 'bool']:
+                        ccdict['book'] = Column(Integer, ForeignKey('books.id'))
+                    cc_classes[row.id] = type(str('custom_column_' + str(row.id)), (Base,), ccdict)

            for cc_id in cc_ids:
                if (cc_id[1] == 'bool') or (cc_id[1] == 'int') or (cc_id[1] == 'float'):
@ -511,6 +524,11 @@ class CalibreDB(threading.Thread):
                                         primaryjoin=(
                                         Books.id == cc_classes[cc_id[0]].book),
                                         backref='books'))
+                elif (cc_id[1] == 'series'):
+                    setattr(Books,
+                            'custom_column_' + str(cc_id[0]),
+                            relationship(books_custom_column_links[cc_id[0]],
+                                         backref='books'))
                else:
                    setattr(Books,
                            'custom_column_' + str(cc_id[0]),
--- a/cps/editbooks.py
+++ b/cps/editbooks.py
@ -867,8 +867,8 @@ def upload():

                # move cover to final directory, including book id
                if has_cover:
+                    new_coverpath = os.path.join(config.config_calibre_dir, db_book.path, "cover.jpg")
                    try:
-                        new_coverpath = os.path.join(config.config_calibre_dir, db_book.path, "cover.jpg")
                        copyfile(meta.cover, new_coverpath)
                        os.unlink(meta.cover)
                    except OSError as e:
--- a/cps/gdrive.py
+++ b/cps/gdrive.py
@ -34,18 +34,17 @@ from flask import Blueprint, flash, request, redirect, url_for, abort
 from flask_babel import gettext as _
 from flask_login import login_required

-try:
-    from googleapiclient.errors import HttpError
-except ImportError:
-    pass
-
 from . import logger, gdriveutils, config, ub, calibre_db
 from .web import admin_required

-
 gdrive = Blueprint('gdrive', __name__)
 log = logger.create()

+try:
+    from googleapiclient.errors import HttpError
+except ImportError as err:
+    log.debug(("Cannot import googleapiclient, using gdrive will not work: %s", err))
+
 current_milli_time = lambda: int(round(time() * 1000))

 gdrive_watch_callback_token = 'target=calibreweb-watch_files'
@ -73,7 +72,7 @@ def google_drive_callback():
        credentials = gdriveutils.Gauth.Instance().auth.flow.step2_exchange(auth_code)
        with open(gdriveutils.CREDENTIALS, 'w') as f:
            f.write(credentials.to_json())
-    except ValueError as error:
+    except (ValueError, AttributeError) as error:
        log.error(error)
    return redirect(url_for('admin.configuration'))

@ -94,8 +93,7 @@ def watch_gdrive():
        try:
            result = gdriveutils.watchChange(gdriveutils.Gdrive.Instance().drive, notification_id,
                               'web_hook', address, gdrive_watch_callback_token, current_milli_time() + 604800*1000)
-            config.config_google_drive_watch_changes_response = json.dumps(result)
-            # after save(), config_google_drive_watch_changes_response will be a json object, not string
+            config.config_google_drive_watch_changes_response = result
            config.save()
        except HttpError as e:
            reason=json.loads(e.content)['error']['errors'][0]
@ -118,7 +116,7 @@ def revoke_watch_gdrive():
                                    last_watch_response['resourceId'])
        except HttpError:
            pass
-        config.config_google_drive_watch_changes_response = None
+        config.config_google_drive_watch_changes_response = {}
        config.save()
    return redirect(url_for('admin.configuration'))

@ -155,7 +153,7 @@ def on_received_watch_confirmation():
                        log.info('Setting up new DB')
                        # prevent error on windows, as os.rename does on exisiting files
                        move(os.path.join(tmpDir, "tmp_metadata.db"), dbpath)
-                        calibre_db.setup_db(config, ub.app_DB_path)
+                        calibre_db.reconnect_db(config, ub.app_DB_path)
            except Exception as e:
                log.exception(e)
        updateMetaData()
--- a/cps/gdriveutils.py
+++ b/cps/gdriveutils.py
@ -36,7 +36,9 @@ try:
    from apiclient import errors
    from httplib2 import ServerNotFoundError
    gdrive_support = True
-except ImportError:
+    importError = None
+except ImportError as err:
+    importError = err
    gdrive_support = False

 from . import logger, cli, config
@ -52,6 +54,8 @@ if gdrive_support:
    logger.get('googleapiclient.discovery_cache').setLevel(logger.logging.ERROR)
    if not logger.is_debug_enabled():
        logger.get('googleapiclient.discovery').setLevel(logger.logging.ERROR)
+else:
+    log.debug("Cannot import pydrive,httplib2, using gdrive will not work: %s", importError)


 class Singleton:
@ -99,7 +103,11 @@ class Singleton:
@Singleton
 class Gauth:
    def __init__(self):
-        self.auth = GoogleAuth(settings_file=SETTINGS_YAML)
+        try:
+            self.auth = GoogleAuth(settings_file=SETTINGS_YAML)
+        except NameError as error:
+            log.error(error)
+            self.auth = None


@Singleton
@ -594,8 +602,12 @@ def get_error_text(client_secrets=None):
    if not os.path.isfile(CLIENT_SECRETS):
        return 'client_secrets.json is missing or not readable'

-    with open(CLIENT_SECRETS, 'r') as settings:
-        filedata = json.load(settings)
+    try:
+        with open(CLIENT_SECRETS, 'r') as settings:
+            filedata = json.load(settings)
+    except PermissionError:
+        return 'client_secrets.json is missing or not readable'
+
    if 'web' not in filedata:
        return 'client_secrets.json is not configured for web application'
    if 'redirect_uris' not in filedata['web']:
--- a/cps/helper.py
+++ b/cps/helper.py
@ -295,15 +295,16 @@ def delete_book_file(book, calibrepath, book_format=None):
            return True, None
        else:
            if os.path.isdir(path):
-                if len(next(os.walk(path))[1]):
-                    log.error("Deleting book %s failed, path has subfolders: %s", book.id, book.path)
-                    return False , _("Deleting book %(id)s failed, path has subfolders: %(path)s",
-                                     id=book.id,
-                                     path=book.path)
                try:
-                    for root, __, files in os.walk(path):
+                    for root, folders, files in os.walk(path):
                        for f in files:
                            os.unlink(os.path.join(root, f))
+                        if len(folders):
+                            log.warning("Deleting book {} failed, path {} has subfolders: {}".format(book.id,
+                                        book.path, folders))
+                            return True, _("Deleting bookfolder for book %(id)s failed, path has subfolders: %(path)s",
+                                            id=book.id,
+                                            path=book.path)
                    shutil.rmtree(path)
                except (IOError, OSError) as e:
                    log.error("Deleting book %s failed: %s", book.id, e)
@ -339,13 +340,13 @@ def update_dir_structure_file(book_id, calibrepath, first_author):
        new_title_path = os.path.join(os.path.dirname(path), new_titledir)
        try:
            if not os.path.exists(new_title_path):
-                os.renames(path, new_title_path)
+                os.renames(os.path.normcase(path), os.path.normcase(new_title_path))
            else:
                log.info("Copying title: %s into existing: %s", path, new_title_path)
                for dir_name, __, file_list in os.walk(path):
                    for file in file_list:
-                        os.renames(os.path.join(dir_name, file),
-                                   os.path.join(new_title_path + dir_name[len(path):], file))
+                        os.renames(os.path.normcase(os.path.join(dir_name, file)),
+                                   os.path.normcase(os.path.join(new_title_path + dir_name[len(path):], file)))
            path = new_title_path
            localbook.path = localbook.path.split('/')[0] + '/' + new_titledir
        except OSError as ex:
@ -356,7 +357,7 @@ def update_dir_structure_file(book_id, calibrepath, first_author):
    if authordir != new_authordir:
        new_author_path = os.path.join(calibrepath, new_authordir, os.path.basename(path))
        try:
-            os.renames(path, new_author_path)
+            os.renames(os.path.normcase(path), os.path.normcase(new_author_path))
            localbook.path = new_authordir + '/' + localbook.path.split('/')[1]
        except OSError as ex:
            log.error("Rename author from: %s to %s: %s", path, new_author_path, ex)
@ -369,8 +370,9 @@ def update_dir_structure_file(book_id, calibrepath, first_author):
            new_name = get_valid_filename(localbook.title) + ' - ' + get_valid_filename(new_authordir)
            path_name = os.path.join(calibrepath, new_authordir, os.path.basename(path))
            for file_format in localbook.data:
-                os.renames(os.path.join(path_name, file_format.name + '.' + file_format.format.lower()),
-                           os.path.join(path_name, new_name + '.' + file_format.format.lower()))
+                os.renames(os.path.normcase(
+                    os.path.join(path_name, file_format.name + '.' + file_format.format.lower())),
+                           os.path.normcase(os.path.join(path_name, new_name + '.' + file_format.format.lower())))
                file_format.name = new_name
        except OSError as ex:
            log.error("Rename file in path %s to %s: %s", path, new_name, ex)
--- a/cps/jinjia.py
+++ b/cps/jinjia.py
@ -107,3 +107,10 @@ def timestamptodate(date, fmt=None):
@jinjia.app_template_filter('yesno')
 def yesno(value, yes, no):
    return yes if value else no
+
+@jinjia.app_template_filter('formatfloat')
+def formatfloat(value, decimals=1):
+    formatedstring = '%d' % value
+    if (value % 1) != 0:
+        formatedstring = ('%s.%d' % (formatedstring, (value % 1) * 10**decimals)).rstrip('0')
+    return formatedstring
--- a/cps/kobo.py
+++ b/cps/kobo.py
@ -19,8 +19,6 @@

 import base64
 import datetime
-import itertools
-import json
 import sys
 import os
 import uuid
@ -267,7 +265,7 @@ def HandleMetadataRequest(book_uuid):

 def get_download_url_for_book(book, book_format):
    if not current_app.wsgi_app.is_proxied:
-        if ':' in request.host and not request.host.endswith(']') :
+        if ':' in request.host and not request.host.endswith(']'):
            host = "".join(request.host.split(':')[:-1])
        else:
            host = request.host
@ -317,8 +315,15 @@ def get_description(book):
 # TODO handle multiple authors
 def get_author(book):
    if not book.authors:
-        return None
-    return book.authors[0].name
+        return {"Contributors": None}
+    if len(book.authors) > 1:
+        author_list = []
+        autor_roles = []
+        for author in book.authors:
+            autor_roles.append({"Name":author.name, "Role":"Author"})
+            author_list.append(author.name)
+        return {"ContributorRoles": autor_roles, "Contributors":author_list}
+    return {"ContributorRoles": [{"Name":book.authors[0].name, "Role":"Author"}], "Contributors": book.authors[0].name}


 def get_publisher(book):
@ -357,7 +362,7 @@ def get_metadata(book):
    book_uuid = book.uuid
    metadata = {
        "Categories": ["00000000-0000-0000-0000-000000000001",],
-        "Contributors": get_author(book),
+        # "Contributors": get_author(book),
        "CoverImageId": book_uuid,
        "CrossRevisionId": book_uuid,
        "CurrentDisplayPrice": {"CurrencyCode": "USD", "TotalAmount": 0},
@ -381,6 +386,7 @@ def get_metadata(book):
        "Title": book.title,
        "WorkId": book_uuid,
    }
+    metadata.update(get_author(book))

    if get_series(book):
        if sys.version_info < (3, 0):
@ -399,7 +405,7 @@ def get_metadata(book):


@kobo.route("/v1/library/tags", methods=["POST", "DELETE"])
-@login_required
+@requires_kobo_auth
 # Creates a Shelf with the given items, and returns the shelf's uuid.
 def HandleTagCreate():
    # catch delete requests, otherwise the are handeld in the book delete handler
@ -434,6 +440,7 @@ def HandleTagCreate():


@kobo.route("/v1/library/tags/<tag_id>", methods=["DELETE", "PUT"])
+@requires_kobo_auth
 def HandleTagUpdate(tag_id):
    shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.uuid == tag_id,
                                              ub.Shelf.user_id == current_user.id).one_or_none()
@ -488,7 +495,7 @@ def add_items_to_shelf(items, shelf):


@kobo.route("/v1/library/tags/<tag_id>/items", methods=["POST"])
-@login_required
+@requires_kobo_auth
 def HandleTagAddItem(tag_id):
    items = None
    try:
@ -518,7 +525,7 @@ def HandleTagAddItem(tag_id):


@kobo.route("/v1/library/tags/<tag_id>/items/delete", methods=["POST"])
-@login_required
+@requires_kobo_auth
 def HandleTagRemoveItem(tag_id):
    items = None
    try:
@ -627,7 +634,7 @@ def create_kobo_tag(shelf):


@kobo.route("/v1/library/<book_uuid>/state", methods=["GET", "PUT"])
-@login_required
+@requires_kobo_auth
 def HandleStateRequest(book_uuid):
    book = calibre_db.get_book_by_uuid(book_uuid)
    if not book or not book.data:
@ -801,7 +808,7 @@ def TopLevelEndpoint():


@kobo.route("/v1/library/<book_uuid>", methods=["DELETE"])
-@login_required
+@requires_kobo_auth
 def HandleBookDeletionRequest(book_uuid):
    log.info("Kobo book deletion request received for book %s" % book_uuid)
    book = calibre_db.get_book_by_uuid(book_uuid)
--- a/cps/oauth_bb.py
+++ b/cps/oauth_bb.py
@ -287,7 +287,7 @@ if ub.oauth_support:
                    flash(_(u"Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error")
        except NoResultFound:
            log.warning("oauth %s for user %d not found", provider, current_user.id)
-            flash(_(u"Not Linked to %(oauth)s.", oauth=oauth_check[provider]), category="error")
+            flash(_(u"Not Linked to %(oauth)s", oauth=provider), category="error")
        return redirect(url_for('web.profile'))


@ -355,4 +355,4 @@ if ub.oauth_support:
    @oauth.route('/unlink/google', methods=["GET"])
    @login_required
    def google_login_unlink():
-        return unlink_oauth(oauthblueprints[1]['blueprint'].name)
+        return unlink_oauth(oauthblueprints[1]['id'])
--- a/cps/reverseproxy.py
+++ b/cps/reverseproxy.py
@ -77,6 +77,7 @@ class ReverseProxied(object):
        servr = environ.get('HTTP_X_FORWARDED_HOST', '')
        if servr:
            environ['HTTP_HOST'] = servr
+            self.proxied = True
        return self.app(environ, start_response)

    @property
--- a/cps/services/SyncToken.py
+++ b/cps/services/SyncToken.py
@ -27,7 +27,10 @@ except ImportError:
    from urllib.parse import unquote

 from flask import json
-from .. import logger as log
+from .. import logger
+
+
+log = logger.create()


 def b64encode_json(json_data):
@ -45,7 +48,8 @@ def to_epoch_timestamp(datetime_object):
 def get_datetime_from_json(json_object, field_name):
    try:
        return datetime.utcfromtimestamp(json_object[field_name])
-    except KeyError:
+    except (KeyError, OSError, OverflowError):
+        # OSError is thrown on Windows if timestamp is <1970 or >2038
        return datetime.min


--- a/cps/services/simpleldap.py
+++ b/cps/services/simpleldap.py
@ -65,7 +65,10 @@ def init_app(app, config):
    app.config['LDAP_GROUP_OBJECT_FILTER'] = config.config_ldap_group_object_filter
    app.config['LDAP_GROUP_MEMBERS_FIELD'] = config.config_ldap_group_members_field

-    _ldap.init_app(app)
+    try:
+        _ldap.init_app(app)
+    except RuntimeError as e:
+        log.error(e)


 def get_object_details(user=None, group=None, query_filter=None, dn_only=False):
--- a/cps/templates/book_edit.html
+++ b/cps/templates/book_edit.html
@ -130,19 +130,20 @@
 	    <input type="number" step="{% if c.datatype == 'float' %}0.01{% else %}1{% endif %}" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}" value="{% if book['custom_column_' ~ c.id]|length > 0 %}{{ book['custom_column_' ~ c.id][0].value }}{% endif %}">
            {% endif %}

-            {% if c.datatype in ['text', 'series'] and not c.is_multiple %}
+            {% if c.datatype == 'text' %}
              <input type="text" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}"
              {% if book['custom_column_' ~ c.id]|length > 0 %}
-                value="{{ book['custom_column_' ~ c.id][0].value }}"
-              {% endif %}>
+              value="{% for column in book['custom_column_' ~ c.id] %}{{ column.value.strip() }}{% if not loop.last %}, {% endif %}{% endfor %}"{% endif %}>
            {% endif %}

-            {% if c.datatype in ['text', 'series'] and c.is_multiple %}
+            {% if c.datatype == 'series' %}
              <input type="text" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}"
              {% if book['custom_column_' ~ c.id]|length > 0 %}
-              value="{% for column in book['custom_column_' ~ c.id] %}{{ column.value.strip() }}{% if not loop.last %}, {% endif %}{% endfor %}"{% endif %}>
+                value="{% for column in book['custom_column_' ~ c.id] %} {{ '%s [%s]' % (book['custom_column_' ~ c.id][0].value, book['custom_column_' ~ c.id][0].extra|formatfloat(2)) }}{% if not loop.last %}, {% endif %}{% endfor %}"
+              {% endif %}>
            {% endif %}

+
            {% if c.datatype == 'enumeration' %}
              <select class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}">
                  <option></option>
@ -157,9 +158,9 @@
            {% endif %}

            {% if c.datatype == 'rating' %}
-              <input type="number" min="1" max="5" step="1" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}"
+              <input type="number" min="1" max="5" step="0.5" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}"
                {% if book['custom_column_' ~ c.id]|length > 0 %}
-                    value="{{ '%d' %  (book['custom_column_' ~ c.id][0].value / 2) }}"
+                    value="{{ '%.1f' %  (book['custom_column_' ~ c.id][0].value / 2) }}"
                {% endif %}>
            {% endif %}
          </div>
--- a/cps/templates/config_edit.html
+++ b/cps/templates/config_edit.html
@ -30,20 +30,20 @@
    <div data-related="gdrive_settings">
      {% if gdriveError %}
        <div class="form-group">
-          <label>
+          <label id="gdrive_error">
            {{_('Google Drive config problem')}}: {{ gdriveError }}
          </label>
        </div>
      {% else %}
        {% if show_authenticate_google_drive and g.user.is_authenticated and config.config_use_google_drive %}
          <div class="form-group required">
-            <a href="{{ url_for('gdrive.authenticate_google_drive') }}" class="btn btn-primary">{{_('Authenticate Google Drive')}}</a>
+            <a href="{{ url_for('gdrive.authenticate_google_drive') }}" id="gdrive_auth" class="btn btn-primary">{{_('Authenticate Google Drive')}}</a>
          </div>
        {% else %}
            {% if show_authenticate_google_drive and g.user.is_authenticated and not config.config_use_google_drive %}
-            <div >{{_('Please hit submit to continue with setup')}}</div>
+            <div >{{_('Please hit save to continue with setup')}}</div>
            {% endif %}
-          {% if not g.user.is_authenticated %}
+          {% if not g.user.is_authenticated and show_login_button %}
          <div >{{_('Please finish Google Drive setup after login')}}</div>
            {% endif %}
          {% if g.user.is_authenticated %}
--- a/cps/templates/detail.html
+++ b/cps/templates/detail.html
@ -174,7 +174,7 @@
            {{ c.name }}:
            {% for column in entry['custom_column_' ~ c.id] %}
              {% if c.datatype == 'rating' %}
-                {{ '%d' % (column.value / 2) }}
+                {{ (column.value / 2)|formatfloat }}
              {% else %}
                {% if c.datatype == 'bool' %}
                  {% if column.value == true %}
@ -182,9 +182,17 @@
                  {% else %}
                    <span class="glyphicon glyphicon-remove"></span>
                  {% endif %}
+                {% else %}
+                {% if c.datatype == 'float' %}
+                  {{ column.value|formatfloat(2) }}
+                {% else %}
+                {% if c.datatype == 'series' %}
+                  {{ '%s [%s]' % (column.value, column.extra|formatfloat(2)) }}
                {% else %}
                  {{ column.value }}
                {% endif %}
+                {% endif %}
+                {% endif %}
              {% endif %}
            {% endfor %}
          {% endif %}
--- a/cps/templates/search_form.html
+++ b/cps/templates/search_form.html
@ -165,7 +165,7 @@
        {% endif %}

        {% if c.datatype == 'rating' %}
-          <input type="number" min="1" max="5" step="1" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}">
+          <input type="number" min="1" max="5" step="0.5" class="form-control" name="{{ 'custom_column_' ~ c.id }}" id="{{ 'custom_column_' ~ c.id }}">
        {% endif %}
        </div>
      {% endfor %}
--- a/cps/templates/stats.html
+++ b/cps/templates/stats.html
@ -25,6 +25,7 @@
    </tr>
  </tbody>
 </table>
+{% if g.user.role_admin() %}
  <h3>{{_('Linked Libraries')}}</h3>
 <table id="libs" class="table">
  <thead>
@ -44,4 +45,5 @@
  {% endfor %}
  </tbody>
 </table>
+{% endif %}
 {% endblock %}
--- a/cps/translations/cs/LC_MESSAGES/messages.po
+++ b/cps/translations/cs/LC_MESSAGES/messages.po
@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive"
 msgstr "Ověřit Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Klikněte na odeslat pro pokračování v nastavení"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/de/LC_MESSAGES/messages.po
+++ b/cps/translations/de/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "Google Drive authentifizieren"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Bitte auf Abschicken drücken, um mit dem Setup fortzufahren"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/es/LC_MESSAGES/messages.po
+++ b/cps/translations/es/LC_MESSAGES/messages.po
@ -1560,7 +1560,7 @@ msgid "Authenticate Google Drive"
 msgstr "Autentificar Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Por favor, pulsa enviar para continuar con la configuración"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/fi/LC_MESSAGES/messages.po
+++ b/cps/translations/fi/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "Autentikoi Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Ole hyvä ja paina lähetä jatkaaksesi asennusta"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/fr/LC_MESSAGES/messages.po
+++ b/cps/translations/fr/LC_MESSAGES/messages.po
@ -1571,7 +1571,7 @@ msgid "Authenticate Google Drive"
 msgstr "Authentification Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Veuillez cliquer sur soumettre pour continuer l’initialisation"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/hu/LC_MESSAGES/messages.po
+++ b/cps/translations/hu/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "Google Drive hitelesítés"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "A beállítás folytatásához kattints a Küldés gombra"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/it/LC_MESSAGES/messages.po
+++ b/cps/translations/it/LC_MESSAGES/messages.po
@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive"
 msgstr "Autenticazione Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Per favore premi invio per proseguire con la configurazione"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/ja/LC_MESSAGES/messages.po
+++ b/cps/translations/ja/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "Googleドライブを認証"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "決定を押して設定を続けてください"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/km/LC_MESSAGES/messages.po
+++ b/cps/translations/km/LC_MESSAGES/messages.po
@ -1558,7 +1558,7 @@ msgid "Authenticate Google Drive"
 msgstr "វាយបញ្ចូលគណនី Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr ""

 #: cps/templates/config_edit.html:47
--- a/cps/translations/nl/LC_MESSAGES/messages.po
+++ b/cps/translations/nl/LC_MESSAGES/messages.po
@ -1558,7 +1558,7 @@ msgid "Authenticate Google Drive"
 msgstr "Google Drive goedkeuren"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Druk op 'Opslaan' om door te gaan met instellen"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/pl/LC_MESSAGES/messages.po
+++ b/cps/translations/pl/LC_MESSAGES/messages.po
@ -1570,7 +1570,7 @@ msgid "Authenticate Google Drive"
 msgstr "Uwierzytelnij Dysk Google"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Kliknij przycisk, aby kontynuować instalację"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/ru/LC_MESSAGES/messages.po
+++ b/cps/translations/ru/LC_MESSAGES/messages.po
@ -1558,7 +1558,7 @@ msgid "Authenticate Google Drive"
 msgstr "Аутентификация Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Пожалуйста, нажмите «Отправить», чтобы продолжить настройку"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/sv/LC_MESSAGES/messages.po
+++ b/cps/translations/sv/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "Autentisera Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Klicka på skicka för att fortsätta med installationen"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/tr/LC_MESSAGES/messages.po
+++ b/cps/translations/tr/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "Google Drive Doğrula"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "Kuruluma devam etmek için Gönder'e tıklayın"

 #: cps/templates/config_edit.html:47
--- a/cps/translations/uk/LC_MESSAGES/messages.po
+++ b/cps/translations/uk/LC_MESSAGES/messages.po
@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive"
 msgstr "Автентифікація Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr ""

 #: cps/templates/config_edit.html:47
--- a/cps/translations/zh_Hans_CN/LC_MESSAGES/messages.po
+++ b/cps/translations/zh_Hans_CN/LC_MESSAGES/messages.po
@ -1557,7 +1557,7 @@ msgid "Authenticate Google Drive"
 msgstr "认证 Google Drive"

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr "请点击提交以继续设置"

 #: cps/templates/config_edit.html:47
--- a/cps/ub.py
+++ b/cps/ub.py
@ -177,20 +177,20 @@ class UserBase:
        return self.check_visibility(constants.DETAIL_RANDOM)

    def list_denied_tags(self):
-        mct = self.denied_tags.split(",")
-        return [t.strip() for t in mct]
+        mct = self.denied_tags or ""
+        return [t.strip() for t in mct.split(",")]

    def list_allowed_tags(self):
-        mct = self.allowed_tags.split(",")
-        return [t.strip() for t in mct]
+        mct = self.allowed_tags or ""
+        return [t.strip() for t in mct.split(",")]

    def list_denied_column_values(self):
-        mct = self.denied_column_value.split(",")
-        return [t.strip() for t in mct]
+        mct = self.denied_column_value or ""
+        return [t.strip() for t in mct.split(",")]

    def list_allowed_column_values(self):
-        mct = self.allowed_column_value.split(",")
-        return [t.strip() for t in mct]
+        mct = self.allowed_column_value or ""
+        return [t.strip() for t in mct.split(",")]

    def __repr__(self):
        return '<User %r>' % self.nickname
@ -478,34 +478,34 @@ def migrate_Database(session):
        ArchivedBook.__table__.create(bind=engine)
    if not engine.dialect.has_table(engine.connect(), "registration"):
        ReadBook.__table__.create(bind=engine)
-        conn = engine.connect()
-        conn.execute("insert into registration (domain, allow) values('%.%',1)")
+        with engine.connect() as conn:
+            conn.execute("insert into registration (domain, allow) values('%.%',1)")
        session.commit()
    try:
        session.query(exists().where(Registration.allow)).scalar()
        session.commit()
    except exc.OperationalError:  # Database is not compatible, some columns are missing
-        conn = engine.connect()
-        conn.execute("ALTER TABLE registration ADD column 'allow' INTEGER")
-        conn.execute("update registration set 'allow' = 1")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE registration ADD column 'allow' INTEGER")
+            conn.execute("update registration set 'allow' = 1")
        session.commit()
    try:
        session.query(exists().where(RemoteAuthToken.token_type)).scalar()
        session.commit()
    except exc.OperationalError:  # Database is not compatible, some columns are missing
-        conn = engine.connect()
-        conn.execute("ALTER TABLE remote_auth_token ADD column 'token_type' INTEGER DEFAULT 0")
-        conn.execute("update remote_auth_token set 'token_type' = 0")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE remote_auth_token ADD column 'token_type' INTEGER DEFAULT 0")
+            conn.execute("update remote_auth_token set 'token_type' = 0")
        session.commit()
    try:
        session.query(exists().where(ReadBook.read_status)).scalar()
    except exc.OperationalError:
-        conn = engine.connect()
-        conn.execute("ALTER TABLE book_read_link ADD column 'read_status' INTEGER DEFAULT 0")
-        conn.execute("UPDATE book_read_link SET 'read_status' = 1 WHERE is_read")
-        conn.execute("ALTER TABLE book_read_link ADD column 'last_modified' DATETIME")
-        conn.execute("ALTER TABLE book_read_link ADD column 'last_time_started_reading' DATETIME")
-        conn.execute("ALTER TABLE book_read_link ADD column 'times_started_reading' INTEGER DEFAULT 0")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE book_read_link ADD column 'read_status' INTEGER DEFAULT 0")
+            conn.execute("UPDATE book_read_link SET 'read_status' = 1 WHERE is_read")
+            conn.execute("ALTER TABLE book_read_link ADD column 'last_modified' DATETIME")
+            conn.execute("ALTER TABLE book_read_link ADD column 'last_time_started_reading' DATETIME")
+            conn.execute("ALTER TABLE book_read_link ADD column 'times_started_reading' INTEGER DEFAULT 0")
        session.commit()
    test = session.query(ReadBook).filter(ReadBook.last_modified == None).all()
    for book in test:
@ -514,11 +514,11 @@ def migrate_Database(session):
    try:
        session.query(exists().where(Shelf.uuid)).scalar()
    except exc.OperationalError:
-        conn = engine.connect()
-        conn.execute("ALTER TABLE shelf ADD column 'uuid' STRING")
-        conn.execute("ALTER TABLE shelf ADD column 'created' DATETIME")
-        conn.execute("ALTER TABLE shelf ADD column 'last_modified' DATETIME")
-        conn.execute("ALTER TABLE book_shelf_link ADD column 'date_added' DATETIME")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE shelf ADD column 'uuid' STRING")
+            conn.execute("ALTER TABLE shelf ADD column 'created' DATETIME")
+            conn.execute("ALTER TABLE shelf ADD column 'last_modified' DATETIME")
+            conn.execute("ALTER TABLE book_shelf_link ADD column 'date_added' DATETIME")
        for shelf in session.query(Shelf).all():
            shelf.uuid = str(uuid.uuid4())
            shelf.created = datetime.datetime.now()
@ -529,31 +529,31 @@ def migrate_Database(session):
    # Handle table exists, but no content
    cnt = session.query(Registration).count()
    if not cnt:
-        conn = engine.connect()
-        conn.execute("insert into registration (domain, allow) values('%.%',1)")
+        with engine.connect() as conn:
+            conn.execute("insert into registration (domain, allow) values('%.%',1)")
        session.commit()
    try:
        session.query(exists().where(BookShelf.order)).scalar()
    except exc.OperationalError:  # Database is not compatible, some columns are missing
-        conn = engine.connect()
-        conn.execute("ALTER TABLE book_shelf_link ADD column 'order' INTEGER DEFAULT 1")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE book_shelf_link ADD column 'order' INTEGER DEFAULT 1")
        session.commit()
    try:
        create = False
        session.query(exists().where(User.sidebar_view)).scalar()
    except exc.OperationalError:  # Database is not compatible, some columns are missing
-        conn = engine.connect()
-        conn.execute("ALTER TABLE user ADD column `sidebar_view` Integer DEFAULT 1")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE user ADD column `sidebar_view` Integer DEFAULT 1")
        session.commit()
        create = True
    try:
        if create:
-            conn = engine.connect()
-            conn.execute("SELECT language_books FROM user")
+            with engine.connect() as conn:
+                conn.execute("SELECT language_books FROM user")
            session.commit()
    except exc.OperationalError:
-        conn = engine.connect()
-        conn.execute("UPDATE user SET 'sidebar_view' = (random_books* :side_random + language_books * :side_lang "
+        with engine.connect() as conn:
+            conn.execute("UPDATE user SET 'sidebar_view' = (random_books* :side_random + language_books * :side_lang "
                     "+ series_books * :side_series + category_books * :side_category + hot_books * "
                     ":side_hot + :side_autor + :detail_random)",
                     {'side_random': constants.SIDEBAR_RANDOM, 'side_lang': constants.SIDEBAR_LANGUAGE,
@ -564,35 +564,29 @@ def migrate_Database(session):
    try:
        session.query(exists().where(User.denied_tags)).scalar()
    except exc.OperationalError:  # Database is not compatible, some columns are missing
-        conn = engine.connect()
-        conn.execute("ALTER TABLE user ADD column `denied_tags` String DEFAULT ''")
-        conn.execute("ALTER TABLE user ADD column `allowed_tags` String DEFAULT ''")
-        conn.execute("ALTER TABLE user ADD column `denied_column_value` DEFAULT ''")
-        conn.execute("ALTER TABLE user ADD column `allowed_column_value` DEFAULT ''")
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE user ADD column `denied_tags` String DEFAULT ''")
+            conn.execute("ALTER TABLE user ADD column `allowed_tags` String DEFAULT ''")
+            conn.execute("ALTER TABLE user ADD column `denied_column_value` String DEFAULT ''")
+            conn.execute("ALTER TABLE user ADD column `allowed_column_value` String DEFAULT ''")
        session.commit()
-    #try:
-    #    session.query(exists().where(User.series_view)).scalar()
-    #except exc.OperationalError:
-    #    conn = engine.connect()
-    #    conn.execute("ALTER TABLE user ADD column `series_view` VARCHAR(10) DEFAULT 'list'")
    try:
        session.query(exists().where(User.view_settings)).scalar()
    except exc.OperationalError:
-        conn = engine.connect()
-        conn.execute("ALTER TABLE user ADD column `view_settings` JSON default '{}'")
-        session.commit()
+        with engine.connect() as conn:
+            conn.execute("ALTER TABLE user ADD column `series_view` VARCHAR(10) DEFAULT 'list'")

    if session.query(User).filter(User.role.op('&')(constants.ROLE_ANONYMOUS) == constants.ROLE_ANONYMOUS).first() \
        is None:
        create_anonymous_user(session)
    try:
        # check if one table with autoincrement is existing (should be user table)
-        conn = engine.connect()
-        conn.execute("SELECT COUNT(*) FROM sqlite_sequence WHERE name='user'")
+        with engine.connect() as conn:
+            conn.execute("SELECT COUNT(*) FROM sqlite_sequence WHERE name='user'")
    except exc.OperationalError:
        # Create new table user_id and copy contents of table user into it
-        conn = engine.connect()
-        conn.execute("CREATE TABLE user_id (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,"
+        with engine.connect() as conn:
+            conn.execute("CREATE TABLE user_id (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,"
                     "nickname VARCHAR(64),"
                     "email VARCHAR(120),"
                     "role SMALLINT,"
@ -605,20 +599,19 @@ def migrate_Database(session):
                     "view_settings VARCHAR,"                     
                     "UNIQUE (nickname),"
                     "UNIQUE (email))")
-        conn.execute("INSERT INTO user_id(id, nickname, email, role, password, kindle_mail,locale,"
-                     # "sidebar_view, default_language, series_view) "
-                     "sidebar_view, default_language) "
+            conn.execute("INSERT INTO user_id(id, nickname, email, role, password, kindle_mail,locale,"
+                     "sidebar_view, default_language, series_view) "
                     "SELECT id, nickname, email, role, password, kindle_mail, locale,"
                     "sidebar_view, default_language FROM user")
-        # delete old user table and rename new user_id table to user:
-        conn.execute("DROP TABLE user")
-        conn.execute("ALTER TABLE user_id RENAME TO user")
+            # delete old user table and rename new user_id table to user:
+            conn.execute("DROP TABLE user")
+            conn.execute("ALTER TABLE user_id RENAME TO user")
        session.commit()

    # Remove login capability of user Guest
    try:
-        conn = engine.connect()
-        conn.execute("UPDATE user SET password='' where nickname = 'Guest' and password !=''")
+        with engine.connect() as conn:
+            conn.execute("UPDATE user SET password='' where nickname = 'Guest' and password !=''")
        session.commit()
    except exc.OperationalError:
        print('Settings database is not writeable. Exiting...')
@ -686,8 +679,6 @@ def init_db(app_db_path):

    app_DB_path = app_db_path
    engine = create_engine(u'sqlite:///{0}'.format(app_db_path), echo=False)
-    # engine.execute("attach database '{0}' as app_settings;".format(app_db_path))
-

    Session = sessionmaker()
    Session.configure(bind=engine)
--- a/cps/uploader.py
+++ b/cps/uploader.py
@ -35,7 +35,7 @@ except ImportError:
    lxmlversion = None

 try:
-    from wand.image import Image
+    from wand.image import Image, Color
    from wand import version as ImageVersion
    from wand.exceptions import PolicyError
    use_generic_pdf_cover = False
@ -116,8 +116,8 @@ def default_meta(tmp_file_path, original_file_name, original_file_extension):
 def pdf_meta(tmp_file_path, original_file_name, original_file_extension):
    doc_info = None
    if use_pdf_meta:
-        doc_info = PdfFileReader(open(tmp_file_path, 'rb')).getDocumentInfo()
-
+        with open(tmp_file_path, 'rb') as f:
+            doc_info = PdfFileReader(f).getDocumentInfo()
    if doc_info:
        author = doc_info.author if doc_info.author else u'Unknown'
        title = doc_info.title if doc_info.title else original_file_name
@ -149,6 +149,9 @@ def pdf_preview(tmp_file_path, tmp_dir):
            img.options["pdf:use-cropbox"] = "true"
            img.read(filename=tmp_file_path + '[0]', resolution=150)
            img.compression_quality = 88
+            if img.alpha_channel:
+                img.alpha_channel = 'remove'
+                img.background_color = Color('white')
            img.save(filename=os.path.join(tmp_dir, cover_file_name))
        return cover_file_name
    except PolicyError as ex:
@ -156,6 +159,7 @@ def pdf_preview(tmp_file_path, tmp_dir):
        return None
    except Exception as ex:
        log.warning('Cannot extract cover image, using default: %s', ex)
+        log.warning('On Windows this error could be caused by missing ghostscript')
        return None


--- a/cps/web.py
+++ b/cps/web.py
@ -1316,7 +1316,7 @@ def advanced_search():
                        db.cc_classes[c.id].value == custom_query))
                elif c.datatype == 'rating':
                    q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
-                        db.cc_classes[c.id].value == int(custom_query) * 2))
+                        db.cc_classes[c.id].value == int(float(custom_query) * 2)))
                else:
                    q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
                        func.lower(db.cc_classes[c.id].value).ilike("%" + custom_query + "%")))
@ -1349,6 +1349,9 @@ def advanced_search_form():
 def get_cover(book_id):
    return get_book_cover(book_id)

+@web.route("/robots.txt")
+def get_robots():
+    return send_from_directory(constants.STATIC_DIR, "robots.txt")

@web.route("/show/<int:book_id>/<book_format>", defaults={'anyname': 'None'})
@web.route("/show/<int:book_id>/<book_format>/<anyname>")
--- a/messages.pot
+++ b/messages.pot
@ -1556,7 +1556,7 @@ msgid "Authenticate Google Drive"
 msgstr ""

 #: cps/templates/config_edit.html:44
-msgid "Please hit submit to continue with setup"
+msgid "Please hit save to continue with setup"
 msgstr ""

 #: cps/templates/config_edit.html:47
--- a/TestSummary.html
+++ b/TestSummary.html
@ -36,17 +36,17 @@
      <div class="col-xs-12 col-sm-6">
      <div class="row">
        <div class="col-xs-6 col-md-6 col-sm-offset-3" style="margin-top:50px;">
-            <p class='text-justify attribute'><strong>Start Time: </strong>2020-06-28 20:44:31</p>
+            <p class='text-justify attribute'><strong>Start Time: </strong>2020-08-14 19:46:42</p>
        </div>
      </div>
      <div class="row">
        <div class="col-xs-6 col-md-6 col-sm-offset-3">
-            <p class='text-justify attribute'><strong>Stop Time: </strong>2020-06-28 21:48:11</p>
+            <p class='text-justify attribute'><strong>Stop Time: </strong>2020-08-14 21:02:08</p>
        </div>
      </div>
      <div class="row">
        <div class="col-xs-6 col-md-6 col-sm-offset-3">
-           <p class='text-justify attribute'><strong>Duration: </strong>53:53 min</p>
+           <p class='text-justify attribute'><strong>Duration: </strong>1h 243 min</p>
        </div>
      </div>
      </div>
@ -570,8 +570,8 @@ AssertionError: False is not true</pre>
    <tr class="result['header']['style']">
        <td>test_edit_books.TestEditBooks</td>
        <td class="text-center">33</td>
-        <td class="text-center">30</td>
-        <td class="text-center">1</td>
+        <td class="text-center">29</td>
+        <td class="text-center">2</td>
        <td class="text-center">0</td>
        <td class="text-center">2</td>
        <td class="text-center">
@ -894,11 +894,33 @@ AssertionError: False is not true</pre>
    
    
    
-        <tr id='pt7.32' class='hiddenRow bg-success'>
+        <tr id='ft7.32' class='none bg-danger'>
            <td>
                <div class='testcase'>test_upload_book_pdf</div>
            </td>
-            <td colspan='6' align='center'>PASS</td>
+            <td colspan='6'>
+                <div class="text-center">
+                    <a class="popup_link text-center" onfocus='blur()' onclick="showTestDetail('div_ft7.32')">FAIL</a>
+                </div>
+                <!--css div popup start-->
+                <div id='div_ft7.32' class="popup_window test_output" style="display:none;">
+                    <div class='close_button pull-right'>
+                        <button type="button" class="close" aria-label="Close" onfocus='this.blur();'
+                                onclick="document.getElementById('div_ft7.32').style.display='none'"><span
+                                aria-hidden="true">&times;</span></button>
+                    </div>
+                    <div class="text-left pull-left">
+                        <pre class="text-left">Traceback (most recent call last):
+  File "/home/matthias/Entwicklung/calibre-web-test/test/test_edit_books.py", line 751, in test_upload_book_pdf
+    self.assertEqual('23390', resp.headers['Content-Length'])
+AssertionError: '23390' != '23427'
+- 23390
+ 23427</pre>
+                    </div>
+                    <div class="clearfix"></div>
+                </div>
+                <!--css div popup end-->
+            </td>
        </tr>
    
    
@ -1070,13 +1092,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye

    <tr class="result['header']['style']">
        <td>test_helper.CalibreHelper</td>
-        <td class="text-center">13</td>
-        <td class="text-center">13</td>
+        <td class="text-center">15</td>
+        <td class="text-center">15</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">
-            <a onclick="showClassDetail('c11', 13)">Detail</a>
+            <a onclick="showClassDetail('c11', 15)">Detail</a>
        </td>
    </tr>

@ -1198,6 +1220,24 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
        </tr>
    
    
+    
+        <tr id='pt11.14' class='hiddenRow bg-success'>
+            <td>
+                <div class='testcase'>test_random_password</div>
+            </td>
+            <td colspan='6' align='center'>PASS</td>
+        </tr>
+    
+    
+    
+        <tr id='pt11.15' class='hiddenRow bg-success'>
+            <td>
+                <div class='testcase'>test_whitespaces</div>
+            </td>
+            <td colspan='6' align='center'>PASS</td>
+        </tr>
+    
+    


    <tr class="result['header']['style']">
@ -1289,13 +1329,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye

    <tr class="result['header']['style']">
        <td>test_ldap.test_ldap_login</td>
-        <td class="text-center">9</td>
-        <td class="text-center">9</td>
+        <td class="text-center">10</td>
+        <td class="text-center">10</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">
-            <a onclick="showClassDetail('c13', 9)">Detail</a>
+            <a onclick="showClassDetail('c13', 10)">Detail</a>
        </td>
    </tr>

@ -1374,6 +1414,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
    
    
        <tr id='pt13.9' class='hiddenRow bg-success'>
+            <td>
+                <div class='testcase'>test_ldap_kobo_sync</div>
+            </td>
+            <td colspan='6' align='center'>PASS</td>
+        </tr>
+    
+    
+    
+        <tr id='pt13.10' class='hiddenRow bg-success'>
            <td>
                <div class='testcase'>test_ldap_opds_download_book</div>
            </td>
@ -1480,13 +1529,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye

    <tr class="result['header']['style']">
        <td>test_login.test_Login</td>
-        <td class="text-center">10</td>
-        <td class="text-center">10</td>
+        <td class="text-center">11</td>
+        <td class="text-center">11</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">
-            <a onclick="showClassDetail('c15', 10)">Detail</a>
+            <a onclick="showClassDetail('c15', 11)">Detail</a>
        </td>
    </tr>

@ -1581,6 +1630,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
        </tr>
    
    
+    
+        <tr id='pt15.11' class='hiddenRow bg-success'>
+            <td>
+                <div class='testcase'>test_robots</div>
+            </td>
+            <td colspan='6' align='center'>PASS</td>
+        </tr>
+    
+    


    <tr class="result['header']['style']">
@ -1813,13 +1871,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye

    <tr class="result['header']['style']">
        <td>test_register.test_register</td>
-        <td class="text-center">6</td>
-        <td class="text-center">6</td>
+        <td class="text-center">7</td>
+        <td class="text-center">7</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">0</td>
        <td class="text-center">
-            <a onclick="showClassDetail('c18', 6)">Detail</a>
+            <a onclick="showClassDetail('c18', 7)">Detail</a>
        </td>
    </tr>

@ -1854,7 +1912,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
    
        <tr id='pt18.4' class='hiddenRow bg-success'>
            <td>
-                <div class='testcase'>test_registering_user</div>
+                <div class='testcase'>test_registering_only_email</div>
            </td>
            <td colspan='6' align='center'>PASS</td>
        </tr>
@ -1863,7 +1921,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
    
        <tr id='pt18.5' class='hiddenRow bg-success'>
            <td>
-                <div class='testcase'>test_registering_user_fail</div>
+                <div class='testcase'>test_registering_user</div>
            </td>
            <td colspan='6' align='center'>PASS</td>
        </tr>
@ -1871,6 +1929,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
    
    
        <tr id='pt18.6' class='hiddenRow bg-success'>
+            <td>
+                <div class='testcase'>test_registering_user_fail</div>
+            </td>
+            <td colspan='6' align='center'>PASS</td>
+        </tr>
+    
+    
+    
+        <tr id='pt18.7' class='hiddenRow bg-success'>
            <td>
                <div class='testcase'>test_user_change_password</div>
            </td>
@ -2578,9 +2645,9 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye

    <tr id='total_row' class="text-center bg-grey">
        <td>Total</td>
-        <td>223</td>
-        <td>215</td>
-        <td>2</td>
+        <td>228</td>
+        <td>219</td>
+        <td>3</td>
        <td>0</td>
        <td>6</td>
        <td>&nbsp;</td>
@ -2610,13 +2677,13 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
          
            <tr>
              <th>Platform</th>
-              <td>Linux 5.3.0-59-generic #53~18.04.1-Ubuntu SMP Thu Jun 4 14:58:26 UTC 2020 x86_64 x86_64</td>
+              <td>Linux 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64</td>
              <td>Basic</td>
            </tr>
          
            <tr>
              <th>Python</th>
-              <td>3.7.5</td>
+              <td>3.8.2</td>
              <td>Basic</td>
            </tr>
          
@ -2730,7 +2797,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
          
            <tr>
              <th>Pillow</th>
-              <td>7.1.2</td>
+              <td>7.2.0</td>
              <td>testCoverEditBooks</td>
            </tr>
          
@ -2742,31 +2809,31 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
          
            <tr>
              <th>lxml</th>
-              <td>4.5.1</td>
+              <td>4.5.2</td>
              <td>TestEditAdditionalBooks</td>
            </tr>
          
            <tr>
              <th>Pillow</th>
-              <td>7.1.2</td>
+              <td>7.2.0</td>
              <td>TestEditAdditionalBooks</td>
            </tr>
          
            <tr>
              <th>rarfile</th>
-              <td>3.1</td>
+              <td>4.0</td>
              <td>TestEditAdditionalBooks</td>
            </tr>
          
            <tr>
              <th>lxml</th>
-              <td>4.5.1</td>
+              <td>4.5.2</td>
              <td>TestEditBooks</td>
            </tr>
          
            <tr>
              <th>Pillow</th>
-              <td>7.1.2</td>
+              <td>7.2.0</td>
              <td>TestEditBooks</td>
            </tr>
          
@ -2788,9 +2855,15 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
              <td>test_ldap_login</td>
            </tr>
          
+            <tr>
+              <th>jsonschema</th>
+              <td>3.2.0</td>
+              <td>test_ldap_login</td>
+            </tr>
+          
            <tr>
              <th>python-ldap</th>
-              <td>3.3.0</td>
+              <td>3.3.1</td>
              <td>test_ldap_login</td>
            </tr>
          
@ -2802,7 +2875,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
          
            <tr>
              <th>SQLAlchemy-Utils</th>
-              <td>0.36.6</td>
+              <td>0.36.8</td>
              <td>test_OAuth_login</td>
            </tr>
          
@ -2814,7 +2887,7 @@ AssertionError: False is not true : Browser-Cache Problem: Old Cover is displaye
 </div>

 <script>
-    drawCircle(215, 2, 0, 6);
+    drawCircle(219, 3, 0, 6);
 </script>

 </div>