lunes, 16 de julio de 2018

Why testing? A Brief Intro to Unit Testing



"Wouldn't be more efficient and secure to perform a battery of tests first,
instead of assembling and driving the vehicle to see what happens? "



Inside the paradigm of Software Testing, there are a variety of techniques that are applied for evaluating
all kinds of aspects from a software. Basically, it's a paradigm that defines methods for evaluating functionality,
quality and performance of a software against a myriad of situations that could come across during the path of its
execution, verifying the reliability of its operation and asserting results, against proposed inputs. Besides this,
to write tests, establishes a skeleton of proofs to be performed against a section of code, allowing reusability and
improvement of these tests for Continous Integration or for single test of separate modules (not very efficient
with large applications).

The purpose of this article is to demonstrate the application of Unit Testing concept, which is part of the
Functional Testing category. In a nutshell, Unit Testing is a type of testing focused on evaluating
functionality for specific sections of code. It could be used to evaluate performance at some point
(time consumed during execution of each section of code), but the focus is, to evaluate the functionality
of a code, according to business requirements expressed as inputs against expected outputs for each test.

Let's take as an example this simple module written in Python called calculator.py:


As a good practice, a separate module called calculation_test.py was created, containing tests for the example:


Basically, this module contains a class called Probe which inherits the class TestCase from unittest module
and for each method defined in Probe, a specific method (assertEqual()) from the inherited class (TestCase())
is called, in order to evaluate the functionality of a specific method from Calc, defining specific inputs
for each method and expected outputs.

The output of calculation_test.py is straightforward:

  • Probe class method executed
  • docstring + test result
  • number of executed tests with time elapsed (calulation.py -> time.sleep(5))
  • final message


In case of Exceptions, the output is different (of course).

For demonstration purposes, a non-existent variable was added for one of the Calc methods. Such scenario represents the evaluation of functionality of the code:



As another demonstration, the expected value for one of the Calc methods was modified in one of the assertEqual() calls. This scenario represents the evaluation of Business Requirements VS Expected Output, asserting that a required Input, will produce an expected Output:

In summary, Software Testing is a crucial step regarding the guarantee of functionality, performance and all
expected behaviours of a software, depending of which approach is applied, and is directly related with the
efficiency for the bug fixing process, in order to reach the expected results for a product.

We could say that, Unit Testing is the keystone of Software Testing (but also the beginning of the process),
in order to evaluate the behaviour of our code (Business Requirements VS Expected Output) and identifying all
possible bugs that could happen during the routine of developing softwares.

miércoles, 21 de marzo de 2018

Lists Are Powerful!

In Python, among a variety of data structures, lists are one of the most powerful method for organizing
data collections. A list is very similar to an Array, like in Java, Perl, and so forth, with the difference that
a list can hold a variety of data types, without restrictions of what type of data is allowed to store on it.

For example:

> l
> ['Python', 3.4, {'Author': 'Guido van Rossum'}, ['Computer Science', 'Statistics', 'Games']]
>

We have: a String, a Floating-point number, a Dictionary and a list (ta-da).

In order to understand the flexibility and power of lists, here's a review of possible operations.

Adding

Items are added at end of a list, using append() built-in or added to whatever index position of the list,
using insert(index,"value"), increasing the list length, and all other element indices will be shifted
one position higher. Also, using extend(), one list can be concatenated to another.

# empty list
> l = []

# append to list
> l.append("Python")
> l.append(3.4)
> l.append({"Author":"Guido van Rossum"})
> l.append(["Data Science","Web Development","Geoprocessing"])
> l
> ['Python', 3.4, {'Author': 'Guido van Rossum'}, ['Data Science', 'Statistics', 'Games']]
> len(l)
> 4

# inserting data at index 2
> l.insert(2,1991)
> l
> ['Python', 3.4, 1991, {'Author': 'Guido van Rossum'}, ['Data Science', 'Statistics', 'Games']]
> len(l)
> 5

# concatenating lists
> l.extend([{"Age":27},"Python Software Foundation"])
> l
> ['Python', 3.4, 1991, {'Author': 'Guido van Rossum'}, ['Data Science', 'Statistics', 'Games'], {'Age': 27}, 'Python Software Foundation']
> len(l)
> 7


Removing

Items can be removed by its name, using remove("name"), or removed via its index, with pop(index_num).

By the way, pop() returns the value removed based on index number, which can be stored into a variable, like the example above.

> l = ["Python","JavaScript","Java","Go","C"]
> l
> ['Python', 'JavaScript', 'Java', 'Go', 'C']

# removing by data
> l.remove("Go")
> l
> ['Python', 'JavaScript', 'Java', 'C']

# removing by index: pop() returns the data which was removed
> output = l.pop(1)
> l
> ['Python', 'Java', 'C']
> output
> 'JavaScript'


Slicing

Slicing is something very powerful, indeed. It can be applied to strings (obtaining substrings or single characters), and also for Tuples.

One thing to keep in mind is that, when slicing a list (or a string) with defined start:end (e.g: [1:4]), the end is not included on the slicing. In Python, Lists (as well as Tuples), are zero-indexed.

For example, slicing l = ["p","y","t","h","o","n"], in order to obtain ['y','t','h']: the slice notation would be l[1:4], including the items with index positions 1,2,3, excluding 0,4,5.

> 
> l = ["p","y","t","h","o","n"]
> l
> ['p', 'y', 't', 'h', 'o', 'n']
> 
> print("* first index:", l[0])
> * first index: p
> 
> print("* from index 0 to index 1:", l[:2])
> * from index 0 to index 1: ['p', 'y']
> 
> print("* from index 2 to end of list:", l[2:])
> * from index 2 to end of list: ['t', 'h', 'o', 'n']
> 
> print("* from index 1 to penultimate index: ", l[1:-1])
> * from index 1 to penultimate index: ['y', 't', 'h', 'o']
> 
> print("* from index 0 to enf of list (incr. by 2): ", l[::2])
> * from index 0 to end of list (incr. by 2): ['p', 't', 'o']
> 
> print("* everything, except last two indexes: ", l[:-2])
> * everything, except last two items: ['p', 'y', 't', 'h']
> 
> print("* only the last two indexes: ", l[-2:])
> * only the last two items: ['o', 'n']
> 
> print("* last index: ", l[-1])
> * last index: n
> 
> print("* list palindrome: ", l[::-1])
> * list palindrome: ['n', 'o', 'h', 't', 'y', 'p']
>


Indexes from Items

Very straightforward. Indexes can be obtained, based on item name.

>
> l = ["Python","C++","Java","Cobol","Scala"]
> l
> ['Python', 'C++', 'Java', 'Cobol', 'Scala']
> 
> idx1 = l.index("C++")
> idx2 = l.index("Scala")
> print("Index for C++:   ", idx1) 
> Index for C++:    1
>
> print("Index for Scala: ", idx2)
> Index for Scala:  4
>


Tricks

The last part of this review, brings some extra examples, plus some tricks.

Special attention for the built-in functions min(), max() and count(), returning the minor character, the greater character (both according to alphabetical order) and counting the number of occurrences of a specific letter.

Also some attention for the built-in function reverse(), which reverses the list, doing the same job as the slice notation [::-1]

One thing to keep in mind, regarding copying lists: never use new_list = old_list, for new_list, holds a reference for old_list, and not a copy. That means, if you remove an item from one list, this item will be dropped from the other.

If you want to copy a list to another, use the slice notation new_list = old_list[:].

> 
> l = ["p","y","t"]
> l
> ['p', 'y', 't']
> 
> l += ["h","o","n"]
> print("* List Increment: ", l)
> * List Increment: ['p', 'y', 't', 'h', 'o', 'n']
> 
> l = l * 2
> print("* List Double: ", l)
> * List Double: ['p', 'y', 't', 'h', 'o', 'n', 'p', 'y', 't', 'h', 'o', 'n']
> 
> l.reverse()
> print("* List Palindrome (reverse): ", l)
> * List Palindrome (reverse): ['n', 'o', 'h', 't', 'y', 'p', 'n', 'o', 'h', 't', 'y', 'p']
> 
> l.sort()
> print("* List Sort:", l)
> * List Sort: ['h', 'h', 'n', 'n', 'o', 'o', 'p', 'p', 't', 't', 'y', 'y']
> 
> print("* Min. Char (Alphabetical): ", min(l))
> * Min. Char (Alphabetical): h
> 
> print("* Max. Char (Alphabetical): ", max(l))
> * Max. Char (Alphabetical): y
> 
> print("* Count 'y':", l.count("y"))
> * Count 'y': 2
>

For questions, doubts or improvements, please, leave a comment :).

Best Regards,
@ivanleoncz

jueves, 22 de febrero de 2018

Flask Login App

In the last post, I showed the basic setup for running a Python Flask application, providing a basic example.

Looking at the image above, seems too simple, relying on SQLite3 and Flask for a large scale application, but the fact is that these components, well configured, can form a robust Back-end. It depends on the way that you tune these components and the infrastructure that you have maintaining your application.

Regarding Infrastructure, for example, you can add a more powerful layer in front of a Python Flask application, using Gunicorn as WSGI instead of Werkzeug and NGINX as reverse proxy server, configuring workers on both applications, plus other configurations that only a Web Server like NGINX can provide.

But that's another story, for another article :).

Today, I bring here a more sophisticated application that use these three components above.

NOTICE: Be aware that this application is just an example. It's not a complete solution, although the concepts used here are used in real scenarios.

So, we have here, A Login Application with:

  • SQLite3 database for authentication (sqlite3)
  • cryptographed cookie session (flask.session)
  • templates with dynamic content (flask.render_template)
  • manipulation of request data (flask.request)
  • route redirection (flask.redirect)
  • protect route access for authenticated users (flask.session)

This application is part of this project, which contains some other examples and prototypes written on Python3, mainly. The whole application including the module explained below is here.

So, let's get started, explaining the main application module and its following code:


#!/usr/bin/python3

""" Login application written in Python Flask. """

from flask import Flask, session, redirect, render_template, request

__version__ = "1.0"
__author__  = "ivanleoncz"

import sqlite3db

app = Flask(__name__)
app.config.from_object("config")

@app.route("/")
@app.route("/index")
def f_index():
    return redirect("/login")

@app.route("/login", methods=['GET','POST'])
def f_login():
    if request.method == "GET":
        return render_template("login.html")
    elif request.method == "POST":
        username = request.form['username']
        password = request.form['password'].encode("utf-8")
        database = sqlite3db.Database()
        login = database.validate_user_pass(username,password)
        if login == 0:
            session["username"] = username
            return redirect("/home")
        elif login == 1 or login == 2:
            fail = "Check credentials, please."
            return render_template("login.html",login_failed=fail)

@app.route("/logout")
def f_logout():
    if "username" in session:
        session.pop("username")
    return redirect("/login")

@app.route("/home")
def f_home():
    if "username" in session:
        return render_template("home.html")
    return redirect("/login")

if __name__ == "__main__":
    app.run(host="localhost",port=8000)

The application has a main route for login (/login), which accepts two methods: GET or POST. Depending of which method is used on the request, the function that lays behind the route will respond in a different way through a condition which evaluates the value from request.method object.

So as you can see, the flask.request module plays an important role for the functionality of the /login route:

  • if request.method == "GET": loads the page (login.html), using the flask.render_template module
  • elif request.method == "POST": processes the form submit sent via login.html

Focusing our attention on the POST condition, the flask.request.form object provides the data submitted via the login.html page, getting the username and password which was provided by the user.

After having the credentials, an object for sqlite3db.Database() class is created, for performing tasks on our SQLite3 database file and uses Python sqlite3 module. sqlite3db is a separate module of our application that has all the engine for the authentication database.

Now that we have the object, we can call validate_user_pass(username,password) method, passing the credentials obtained from the POST request.

  • If method returns 0: the authentication was performed successfully and a cookie session is generated, signed with the app secret key and stored on the web browser. This app secret key resides in a separate file which is incorporated to the application at the beginning of the module (app.config.from_object("config")). After performing this process, the user is redirected to /home route and the cookie is stored on the web browser, being valid until the browser windows closes.
  • If method return 1 or 2: the template for /login is reloaded, passing a value for a variable that is present on the template login.html. The variable is already written inside the template, but since that there's no value associated with the variable, no text is loaded. Jinja2 is the template engine that populates this variable and can perform other functionalities regarding HTML templates.

Some routes of the application are protected with an if statement, which evaluates if there is a key with username data stored on flask.session object. Flask uses the app secret key for its calculation in order to validate the signed cookie on the web browser VS the data stored on flask.session object. If no valid data is found, the user is redirected to the /login route.

And finally, I'm using the host and port parameters for flask.run() method, where I can set any hostname, DNS, IP Address or even wildcard ip (0.0.0.0) for configuring the application port to listen on all interfaces from the host.

That's it. For more information and instructions for running this application, take a look on the links at the beginning of the article.

Any questions, please, let me know: I'd be glad to help :).

Best Regards,
@ivanleoncz

domingo, 14 de enero de 2018

A Simple Flask Application


Flask is a widely known microframework based on Python, that uses Werzeug as WSGI and Jinja2 as template engine, both carrying a lot of functionalities and tools.

In this article, I'm going to give a briefing about a basic structure of a Flask application, considering the setup of Virtualenv in order to isolate the environment + packages that are specifically used by your application.


Setting Up The Environment

Before moving to the examples, let's first build the essential environment for a Flask application.

 $ virtualenv -p python3 flask_example
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in flask_example/bin/python3
Also creating executable in flask_example/bin/python
Installing setuptools, pip...done.

 $ source flask_example/bin/activate
 (flask_example) $ cd flask_example/
 (flask_example) flask_example $ pip3 install flask

 (flask_example) flask_example $ python3
Python 3.4.3 (default, Nov 17 2016, 01:08:31) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import flask
>>> dir(flask)
['Blueprint', 'Config', 'Flask', 'Markup', 'Request', 'Response', 'Session', 
'__builtins__', '__cached__', '__doc__','__file__', '__loader__', '__name__',
'__package__', '__path__', '__spec__', '__version__', '_app_ctx_stack', '_compat',
 '_request_ctx_stack', 'abort', 'after_this_request', 'app', 'appcontext_popped', 
'appcontext_pushed', 'appcontext_tearing_down', 'before_render_template', 'blueprints', 
'cli', 'config', 'copy_current_request_context', 'ctx', 'current_app', 'escape', 'flash', 
'g', 'get_flashed_messages', 'get_template_attribute', 'globals', 'got_request_exception', 
'has_app_context', 'has_request_context', 'helpers', 'json', 'json_available', 'jsonify', 
'make_response', 'message_flashed','redirect', 'render_template', 'render_template_string', 
'request', 'request_finished', 'request_started', 'request_tearing_down', 'safe_join', 
'send_file', 'send_from_directory', 'session', 'sessions', 'signals', 'signals_available', 
'stream_with_context', 'template_rendered', 'templating', 'url_for', 'wrappers']


        

Basic App Structure

After setting up our environment, we can build a simple application.

Here are the explanations about the code above:

  • Line 1: determines the script's ability to be executed like an standalone executable
  • Line 3: importing Flask class from flask package
  • Line 5: defining namespace (where it will look for modules, templates, images, etc.)
  • Line 7: for each route, you must define a @app.route decorator
  • Line 8: and for each route, you must define a function
  • Line 9: content returned after successful request
  • Line 11: the app cannot be executed via import (as a module)
  • Line 12: running the app

After writing the modifications to your application, you can run it like a Python script ($ python3 app.py or $ ./app.py) and that's it: you have a Flask App working under a virtual environment.

A simple $ curl http://127.0.0.1:5000/ will show you that it's working :).

A more sophisticated application will be discussed on future posts.

Best Regards,
@ivanleoncz

Why testing? A Brief Intro to Unit Testing

"Wouldn't be more efficient and secure to perform a battery of tests first, instead of assembling and dri...