Vocabulary Scholar docs

I expect the reader to be familiar with node.js environment.

Parts

Web application

Build with React

Mobile application (android only)

Build with React Native

Server application

API for web and mobile applications. Build on node.js, using external mongoDB database as secondary dictionary and for logging errors.

User guide

Settings

Before running the app/server make sure you have all the required settings set. For example the url where the API is running and keys for signing in to Google Drive.

Google Drive will also need app verification in your Google developer account, otherwise it will show warning when user's trying to sing in.

Web app

file settings.js

Mobile app

file .env

Server app

file .env

Environment

Requirements: node, npm (some parts might use yarn, but you can use npm instead).

Standard node.js installation includes also npm. After installation make sure you have the commands node and npm in PATH, for example run node -v.

Versions used for development: node v10.15.0, npm 6.4.1

3-rd party packages

A lot of packages is used, please check package.json

Build

Web app and User Guide can be build on your pc.

The prefred way to build the mobile app is to use Expo. It will use external service to build the app and then you can download the result.

Server does not have build, it's supposed to just start and run all the time.

For build and all other scripts see section "Scripts".

Start mobile app

For development purposes you can start unoptimized version of the app. Run expo start to start the development server (check section "Scripts"), then open the app in your device (using Expo app) or open device simulator in Android Studio.

Publish

To publish mobile app see Expo docs.

Web app and User Guide can be hosted on some static hosting - after they are build of course. But routing might not work entirely, it's better to use hosting that supports Single Page Applications.

Performing the build process and then uploading on the hosting manually might became tedious, so it's wise to choose platform that can automatically build the app and then serve the build version. I used Heroku for that - after pushing to github, I have travis to test the code, if it passes, then Heroku builds the app and updates the deployed version.

For server you will need some hosting that supports node.js apps, static hosting will not suffice, for example Heroku, I used Microsoft Azure.

For hosting Wodnet database that can be downloaded into mobile app, I chose to host it on my private hosting, because the dictionary has about 35MB and both Heroku and MS Azure are in free versions, I would not like to see them overloaded.

Scripts

Web application

Bootstrapped with create-react-app.

Available scripts:

npm run build-css builds css

npm run watch-css watches css for changes and then builds css

npm run start-js starts the app

npm start starts the app and also watches for css changes

npm run build-js builds js

npm run build builds the app

npm test runs unit tests (using jest)

npm run eject ejects the app from create-react-app environment, gives you access to all the configuration (you should do that only if you know what you're doing)

npm run generate-sample runs sample-data-generator to generate random user data - usefull to test how the app handles different amnount of data

npm run cypress:open opens cypress testing framework for end-to-end tests (make sure the app is running locally)

npm run cypress:run runs cypress end-to-end tests using headless browser (make sure the app is running locally)

npm run prettier runs prettier on each .js file

Mobile application

Create with Expo.

You will need to install expo-cli. You can install in locally and then run the scripts below or the preferred way is to install it globally and then you can run directly for example expo start. For all supported commands using expo see expo-cli docs.

Available scripts (using expo-cli directly is the preferred way):

npm start starts local server for the app and gives you a URL to it

npm test runs tests

npm run android build a standalone APK for your project

npm run prettier runs prettier on each .js file

Server

npm run develop starts the server with nodemon tool for better developing experience

npm run parseWordnet runs file parseWordnet.js to parse Wordnet dictionary to use it with the server

npm run saveWordnet runs file saveWordnetDatabase.js to save Wordnet dictionary to mongoDB

npm start starts the server

User guide

Bootstrapped with create-react-app.

Available scripts:

npm start starts the app

npm run build builds the app

Storage

If Google Drive is connected, then it's used as the storage, otherwise inner memory is used, that is Local Storage for web app and device's memory for mobile app.

Data are saved with every change, but max once per second.

Google Drive

Make sure to have all the settings set - see section "Settings".

See "API" - "3rd-party services" - "Google Drive" for more info.

Local Storage

Depending on web browser local Storage has different size, but it's at least 10MB and that should be more than enough - see "Data size" below.

Data size

About 1MB for 1000 words. You can use Sample Data Generator for such details.

In mobile app user can download Wordnet dictionary, the dictionary has about 35MB.

SQLite database

Used to store downloaded dictionary.

Table entries. Stored in pairs id and value, where id is id of the word and value is JSON string containing the word definition.

Value should be divided into more attributes, but I decided to store only one string because of performance issues I had with the dictionary (contains about 150 thousands of entries).

App structure

I only document web and mobile apps. Server and User Guide are quite small.

Screens

All app pages are in folder screens, routing rules are in file routes.js

Components

Some components that are specific for only one page are located with that page in its folder.

I followed the standard separation of components to presentational and container components as is preferred in react apps.

Presentational

Also called "dumb" as they should only concern with how things look, not what they do.

Located in folder components.

Container

Also called smart, they contain most of the logic, usually connected to store. They do not care (much) about looks.

Located in folder containers.

There are alo some High order components - starting with "with", for example WithRedirect.

State manager Redux

Flux architecture, where the app state is saved in store.

Store is folder storage, actions are in folder actions, reducers in folder reducers.

Other folders

Api and gapi - access to APIs, gapi stands for Google Drive API. See section "API" and "Storage" for more details.

(mobile only) database - SQLite database in mobile app for storing dictionary. See section "Storage" for more details.

(web only) scss - styling files in sass. See section "Styling" for more details.

API

Folders api (communicates with Vocabulary Scholar Server) and gapi (communicates with Google Drive).

API is documented in Swagger, which is available online on server. Oxford Dictionaries have documentation in Swagger as well.

Server

Services: word definition, word search, word translation, list supported translation languages, sample lists info, download sample list, dictionaries info, download dictionary, log errors into database.

Responses from Oxford Dictionareis and Microsoft Azure are simply send through usually without any changes.

Cross-origin resource sharing (CORS)

The main service for the app is Oxford Dictionaries, but it does not support CORS, so it cannot be accessed from web browser, that's why server was necessary from the very beginning.

3-rd party services Oxford Dictionaries and Microsoft Azure are not free, so I use CORS on the server to make sure the traffic is from vocabulary-scholar apps.

Sample lists

Because Vocabulary Scholar apps are empty at the beginning I prepared some lists of common words that can be easily imported into the app. Server provides details about those lists and make them available to download.

Error logs

In case there is a critical error in the apps, it's send to the server, which is connected to mongoDB database where the error is logged.

API end-points

"/" no content

"/api-docs" shows API documentation from Swagger

"/dictionaries/:filename" get dictionary

"/errors" log errors

"/lists" get available sample lists

"/lists/:filename" get sample lists

"/search/:query" search word

"/translate" get query translation

"/translate/languages" get supported translation languages

"/entries/:word" get word definition

3-rd party services

Services used: Oxford Dictionaries, Google Drive, Microsoft Azure, mLab

Each service requires registration and obtaining a access key. Also each service can be used for free.

Oxford Dictionaries

Used for: word definition, word search.

API documentation

Paid service, using just free version, which of course has significant limits in usage. In case limit is reached the server will use Wordnet dictionary instead. Wordnet dictionary is free for use and stored in mongoDB database.

Word search can be very resource consuming when we send requests to the API while user types the word, that's why there is 2 seconds delay in the typing to prevent wasting the free limits.

Google Drive

Used for: storing and reading app data.

Besides access keys Google Drive will also require verification of the usage, that needs to be set in your Google developer account.

Used scopes:

In web app all it takes is to include Google Drive script.

In mobile app it's more complicated: after user signs is, the app will get two tokens - main token for all Google Drive communication and refresh token to refresh the main token if it has expired.

Microsoft Azure

Used for: word translation, list supported translation languages.

Paid service, using just free version, which of course has significant limits in usage.

Styling

Web application

Using standard separate CSS files.

Sass

I have my own simple sass framework. Includes CSS normalize. Grid, spacing etc. are generated dynamically. Frameword settings and colors are in separate setting files.

For clean codebase I follow ITCSS and BEM.

Material UI

To focus more on the functionality I use Material UI for React. It has lots of components ready like checkbox or button. In file App.js is created theme for Material UI to have the same colors as sass.

Mobile application

In react native we cannot use separate CSS files, instead we use CSS-in-JS. CSS is written in JS files as JS objects, for that the syntax is a bit different than CSS.

One of the most popular ways how to style React components is styled components.

I also use something like Material UI in web application - it's react-native-elements but that's not so extensive as Material UI, so I also use other packages for icons, modals..., see dependencies in package.json

Styled components

Global styling settings are in stylesTheme.js and global styled components are in stylesComponents.js

Specific styled components are in the corresponding component files at the bottom.

Codebase

Though I work on the app alone, so it's not that important, I strive to keep the codebase clean, that is using modern technics and various tools to keep code according to standards.

Using ES6 with Babel transpiler to support old browsers. Various linters (ESlint, ESlint-react, stylelint) and Prettier.

I'm versioning in git, to make sure no code errors will be saved in git, I'm using husky to automatically run eslint before commit. If eslint finds a problem the commit is rejected.

And I have my code editor configured to run prettier with each file save.

In web app I made also unit and few end-2-end tests. Check section "Scripts" for info how to run tests. Tests are in __tests__ folder, tests for cypress are in cypress/integration.

Supported browsers and devices

Web application

Check package.json - browserslist

Tested in Google Chrome v73 and Firefox v66, Windows 10 64bit. But should work fine in all major browsers regardless OS, IE11+.

Mobile application

React native allows multiplatform development, but I focused only on Android. Anyway just a few parts of the app is Android-specific so it shouldn't be difficult to tweak the code a bit and release iOS version.

Supported Andriod version 5.0+ (API 21+)

Testing on real device - Android 5.1 (API 22), model Lenovo P70-A.

Testing on Android Studio simulator - Android 7.0 (API 24), Android 8.1 (API 27).

Sample data generator

Generates sample data that users might have in their app. Usefull to know the amount of data app stores and to test how app handles with different amont of data.

Folder sample-data-generator in web app.

Samples are randomly generated from 10,000 words (file english-words.js). To tweak the settings check the entry file generator.js there are settings at the top. Output is save in the web app folder as sample-data.json