Tuesday, February 18, 2020

Running Python Web-Servers

There are basically 2 ways which I have learned so far
1. You can run internal python web server
     python.exe -m http.server <8888> --port number is optional.. default is 8000

2. Using Flash web server, and to run that simply call python.exe <python file of flash>. please see below

PS C:\Mandar\Projects\RaterMigration\IhExpressToCogitateRater> python.exe app.py
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 629-182-235
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [30/May/2020 18:11:37] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [30/May/2020 18:11:37] "GET /static/css/styles.css HTTP/1.1" 404 -
127.0.0.1 - - [30/May/2020 18:11:38] "GET /static/img/favicon.ico HTTP/1.1" 404 -
at No comments:

Thursday, February 13, 2020

Flask & Model Connection 

import os
from flask import Flask, flash, request, redirect, url_for
from werkzeug.utils import secure_filename
import numpy as np
import tensorflow as tf
​
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.xception import (
    Xception, preprocess_input, decode_predictions)
​
from tensorflow.keras.models import load_model
​
UPLOAD_FOLDER = '.'
ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg', 'gif'}
​
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
​
    
​
# Refactor above steps into reusable function
def predict(image_path):
    # Load the Xception model
    # https://keras.io/applications/#xception
    model = load_model('xception.h5')
​
    # Default Image Size for Xception
    image_size = (299, 299)
​
    """Use Xception to label image"""
    img = image.load_img(image_path, target_size=image_size)
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)
    
    predictions = model.predict(x)
    #plt.imshow(img)
    
    result = decode_predictions(predictions, top=3)[0]
    print('Predicted:', result)
    return result
​
def allowed_file(filename):
    return '.' in filename and \
           filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
​
​
def classify(filename):
    result = predict(filename)
    #os.remove(filename)
    strTable = '<table>'
    for r in result:
        strTable += '<tr>'
        strTable += '<td>' + str(r[1]) + '</td>'
        strTable += '<td>' + str(r[2]) + '</td>'
        strTable += '</tr>'
    strTable += '</table>'
    strHTML = f'Successfully uploaded {filename} <BR><BR> Prediction results:<BR> {strTable}'
    os.remove(filename)
    return strHTML
​
@app.route('/', methods=['GET', 'POST'])
def upload_file():
    if request.method == 'POST':
        # check if the post request has the file part
        if 'file' not in request.files:
            flash('No file part')
            return redirect(request.url)
        file = request.files['file']
        # if user does not select file, browser also
        # submit an empty part without filename
        if file.filename == '':
            flash('No selected file')
            return redirect(request.url)
        if file and allowed_file(file.filename):
            filename = secure_filename(file.filename)
            file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
            #return redirect(url_for('classify',filename=filename))
            return classify(filename)
​
    return '''
    <!doctype html>
    <title>Upload new File</title>
    <h1>Upload new File</h1>
    <form method=post enctype=multipart/form-data>
      <input type=file name=file>
      <input type=submit value=Upload>
    </form>
    '''
​
​
​
if __name__ == "__main__":
    
    app.run(debug=True)
at No comments:

Thursday, February 6, 2020

Deploying into Heroku


user following video as a referance
https://www.youtube.com/watch?v=FWU_tJqr1Po&list=LLZ4jBpZoYcbYNQAOnuV_edg

Install Heroku CLI from
https://devcenter.heroku.com/articles/heroku-cli
https://devcenter.heroku.com/articles/heroku-cli#download-and-install

Go to command prompt or terminal from VS Code and type heroku login

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> heroku login
heroku: Press any key to open up the browser to login or q to exit:

Once you press any key, it will open a browser and ask you to login to heroku website, once you login successfully, at terminal following message will come

S D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> heroku login
heroku: Press any key to open up the browser to login or q to exit:
Opening browser to https://cli-auth.heroku.com/auth/cli/browser/fe365a7d-34e4-4e81-b4c2-f0a1a6ebb774
Logging in... done
Logged in as mandargogate@gmail.com
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI>

Next step is to create proc file

for that open following url and go to gunicorn web server section in that
https://flask-socketio.readthedocs.io/en/latest/

and copy following code from webpage in new file and rename that as Procfile

gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 module:app

Here module:app is nothing but main app name, in our case app name is app itself

app = Flask(__name__, static_url_path='/static')

& Module is application file name, so in our case its app only

web: gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 app:app

So, the final content of Profile file is as above

then do pip install gunicorn as below

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> pip install gunicorn
Collecting gunicorn
  Downloading https://files.pythonhosted.org/packages/69/ca/926f7cd3a2014b16870086b2d0fdc84a9e49473c68a8dff8b57f7c156f43/gunicorn-20.0.4-py2.py3-none-any.whl (77kB)
     |████████████████████████████████| 81kB 2.6MB/s
Requirement already satisfied: setuptools>=3.0 in c:\users\magogate\appdata\local\programs\python\python38-32\lib\site-packages (from gunicorn) (41.2.0)
Installing collected packages: gunicorn
Successfully installed gunicorn-20.0.4
WARNING: You are using pip version 19.2.3, however version 20.0.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Also, install gevent-websocket [may not be needed]

Now, comment debu=true in app.py file

if __name__ == "__main__":
    # app.run(debug=True)
    app.run()

Now, create requrements.txt file
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> pip freeze > requirements.txt
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI>

Now commit the changes to repository

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> git add .
fatal: not a git repository (or any of the parent directories): .git

Here we got an error, so I manually marked that folder as a git repository using Tortoise Git GUI.. right click and then select the appropriate option

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> git add .
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI>

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> git commit -m "API for accidents and population data for all cities in usa for project3"
[master (root-commit) ecb0297] API for accidents and population data for all cities in usa for project3
 5 files changed, 82 insertions(+)
 create mode 100644 Procfile
 create mode 100644 __pycache__/dao.cpython-38.pyc
 create mode 100644 app.py
 create mode 100644 dao.py
 create mode 100644 requirements.txt

Add heroku git remote to local repository by specifying newly created app

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> heroku git:remote -a usacity-accidents-population
set git remote heroku to https://git.heroku.com/usacity-accidents-population.git
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI>

Now, final step is to push this code from local repo to heroku repo

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> git push heroku master
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 4 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 2.61 KiB | 1.31 MiB/s, done.
Total 8 (delta 0), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Python app detected
remote: -----> Installing python-3.6.10
remote: -----> Installing pip
remote: -----> Installing SQLite3
remote: Sqlite3 successfully installed.
remote: -----> Installing requirements with pip

Now, after deploying when I validated it using https://usacity-accidents-population.herokuapp.com/ it gave an error for websocket..

Now, this pip install did not work for me and it failed.. and looks like for same reason after deploying the app, it was giving that websocket error...

So, finally I changed Procfile as follows
web: gunicorn -w 1 app:app

and committed and re-deployed the code and it started working -- https://usacity-accidents-population.herokuapp.com/city

PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> git add .
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> git commit -m "API for accidents and population data for all cities in usa for project3"
[master 4abc4d9] API for accidents and population data for all cities in usa for project3
 1 file changed, 1 insertion(+), 1 deletion(-)
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI> heroku git:remote -a usacity-accidents-population
set git remote heroku to https://git.heroku.com/usacity-accidents-population.git
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI>  git push heroku master
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 334 bytes | 334.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Python app detected
remote: -----> Need to update SQLite3, clearing cache
remote: -----> Installing python-3.6.10
remote: -----> Installing pip
remote: -----> Installing SQLite3
remote: Sqlite3 successfully installed.
remote: -----> Installing requirements with pip
remote:        Collecting astroid==2.3.3 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 1))
remote:          Downloading https://files.pythonhosted.org/packages/ad/ae/86734823047962e7b8c8529186a1ac4a7ca19aaf1aa0c7713c022ef593fd/astroid-2.3.3-py3-none-any.whl (205kB)
remote:        Collecting beautifulsoup4==4.8.2 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 2))
remote:          Downloading https://files.pythonhosted.org/packages/cb/a1/c698cf319e9cfed6b17376281bd0efc6bfc8465698f54170ef60a485ab5d/beautifulsoup4-4.8.2-py3-none-any.whl (106kB)
remote:        Collecting bs4==0.0.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 3))
remote:          Downloading https://files.pythonhosted.org/packages/10/ed/7e8b97591f6f456174139ec089c769f89a94a1a4025fe967691de971f314/bs4-0.0.1.tar.gz
remote:        Collecting certifi==2019.11.28 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 4))
remote:          Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
remote:        Collecting chardet==3.0.4 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 5))
remote:          Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
remote:        Collecting Click==7.0 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 6))
remote:          Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
remote:        Collecting colorama==0.4.3 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 7))
remote:          Downloading https://files.pythonhosted.org/packages/c9/dc/45cdef1b4d119eb96316b3117e6d5708a08029992b2fee2c143c7a0a5cc5/colorama-0.4.3-py2.py3-none-any.whl
remote:        Collecting dnspython==1.16.0 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 8))
remote:          Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB)
remote:        Collecting Flask==1.1.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 9))
remote:          Downloading https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl (94kB)
remote:        Collecting gunicorn==20.0.4 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 10))
remote:          Downloading https://files.pythonhosted.org/packages/69/ca/926f7cd3a2014b16870086b2d0fdc84a9e49473c68a8dff8b57f7c156f43/gunicorn-20.0.4-py2.py3-none-any.whl (77kB)
remote:        Collecting idna==2.8 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 11))
remote:          Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
remote:        Collecting isort==4.3.21 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 12))
remote:          Downloading https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl (42kB)
remote:        Collecting itsdangerous==1.1.0 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 13))
remote:          Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
remote:        Collecting Jinja2==2.10.3 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 14))
remote:          Downloading https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl (125kB)
remote:        Collecting lazy-object-proxy==1.4.3 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 15))
remote:          Downloading https://files.pythonhosted.org/packages/0b/dd/b1e3407e9e6913cf178e506cd0dee818e58694d9a5cd1984e3f6a8b9a10f/lazy_object_proxy-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (55kB)
remote:        Collecting MarkupSafe==1.1.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 16))
remote:          Downloading https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
remote:        Collecting mccabe==0.6.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 17))
remote:          Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
remote:        Collecting numpy==1.18.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 18))
remote:          Downloading https://files.pythonhosted.org/packages/62/20/4d43e141b5bc426ba38274933ef8e76e85c7adea2c321ecf9ebf7421cedf/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl (20.1MB)
remote:        Collecting pandas==1.0.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 19))
remote:          Downloading https://files.pythonhosted.org/packages/08/ec/b5dd8cfb078380fb5ae9325771146bccd4e8cad2d3e4c72c7433010684eb/pandas-1.0.1-cp36-cp36m-manylinux1_x86_64.whl (10.1MB)
remote:        Collecting pylint==2.4.4 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 20))
remote:          Downloading https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl (302kB)
remote:        Collecting pymongo==3.10.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 21))
remote:          Downloading https://files.pythonhosted.org/packages/49/01/1da7d3709ea54b3b4623c32d521fb263da14822c7d9064d7fd9eeb0b492c/pymongo-3.10.1-cp36-cp36m-manylinux1_x86_64.whl (450kB)
remote:        Collecting python-dateutil==2.8.1 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 22))
remote:          Downloading https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227kB)
remote:        Collecting pytz==2019.3 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 23))
remote:          Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
remote:        Collecting requests==2.22.0 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 24))
remote:          Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)
remote:        Collecting six==1.13.0 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 25))
remote:          Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
remote:        Collecting soupsieve==1.9.5 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 26))
remote:          Downloading https://files.pythonhosted.org/packages/81/94/03c0f04471fc245d08d0a99f7946ac228ca98da4fa75796c507f61e688c2/soupsieve-1.9.5-py2.py3-none-any.whl
remote:        Collecting urllib3==1.25.8 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 27))
remote:          Downloading https://files.pythonhosted.org/packages/e8/74/6e4f91745020f967d09332bb2b8b9b10090957334692eb88ea4afe91b77f/urllib3-1.25.8-py2.py3-none-any.whl (125kB)
remote:        Collecting Werkzeug==0.16.0 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 28))
remote:          Downloading https://files.pythonhosted.org/packages/ce/42/3aeda98f96e85fd26180534d36570e4d18108d62ae36f87694b476b83d6f/Werkzeug-0.16.0-py2.py3-none-any.whl (327kB)
remote:        Collecting wrapt==1.11.2 (from -r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 29))
remote:          Downloading https://files.pythonhosted.org/packages/23/84/323c2415280bc4fc880ac5050dddfb3c8062c2552b34c2e512eb4aa68f79/wrapt-1.11.2.tar.gz
remote:        Collecting typed-ast<1.5,>=1.4.0; implementation_name == "cpython" and python_version < "3.8" (from astroid==2.3.3->-r /tmp/build_5b3c3bf124986455f9993f40796065c0/requirements.txt (line 1))
remote:          Downloading https://files.pythonhosted.org/packages/90/ed/5459080d95eb87a02fe860d447197be63b6e2b5e9ff73c2b0a85622994f4/typed_ast-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (737kB)
remote:        Installing collected packages: lazy-object-proxy, wrapt, six, typed-ast, astroid, soupsieve, beautifulsoup4, bs4, certifi, chardet, Click, colorama, dnspython, itsdangerous, Werkzeug, MarkupSafe, Jinja2, Flask, gunicorn, idna, isort, mccabe, numpy, pytz, python-dateutil, pandas, pylint, pymongo, urllib3, requests
remote:          Running setup.py install for wrapt: started
remote:            Running setup.py install for wrapt: finished with status 'done'
remote:          Running setup.py install for bs4: started
remote:            Running setup.py install for bs4: finished with status 'done'
remote:        Successfully installed Click-7.0 Flask-1.1.1 Jinja2-2.10.3 MarkupSafe-1.1.1 Werkzeug-0.16.0 astroid-2.3.3 beautifulsoup4-4.8.2 bs4-0.0.1 certifi-2019.11.28 chardet-3.0.4 colorama-0.4.3 dnspython-1.16.0 gunicorn-20.0.4 idna-2.8 isort-4.3.21 itsdangerous-1.1.0 lazy-object-proxy-1.4.3 mccabe-0.6.1 numpy-1.18.1 pandas-1.0.1 pylint-2.4.4 pymongo-3.10.1 python-dateutil-2.8.1 pytz-2019.3 requests-2.22.0 six-1.13.0 soupsieve-1.9.5 typed-ast-1.4.1 urllib3-1.25.8 wrapt-1.11.2
remote:
remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote:
remote: -----> Compressing...
remote:        Done: 83.8M
remote: -----> Launching...
remote:        Released v4
remote:        https://usacity-accidents-population.herokuapp.com/ deployed to Heroku
remote:
remote: Verifying deploy... done.
To https://git.heroku.com/usacity-accidents-population.git
   ecb0297..4abc4d9  master -> master
PS D:\GeorgiaTech\ClassWork\Project3_HerokuAPI>
at No comments:
Subscribe to: Posts (Atom)

All about CSS

From book HTML & CSS - Design and Build Websites - Jon Duckett CSS works by associating rules with HTML elements. These rules govern how...