User Tools

Site Tools


start

This is an old revision of the document!


POSTDOCTORAL RESERACH

Scraping and mining twitter streams

Scraping and mining Dezeen articles

  • with scrapy

The setup: python3 in the conda environment

$ conda create -n bots python=3.4 # create a virtual environment named "bots"
$ source activate bots # activate the environment; check if active: conda info --envs
$ conda install -n bots -c conda-forge scrapy # install scrapy for the named environment

Run scrapy directly from the shell:

$ scrapy startproject dezeen # start a project
====== DOCTORAL RESEARCH ======


//>>>>>>>>>>>>>>>>>>>!>!>>> // * // * // * // >>>>>>>>> // // ? // ? !! >>>>>

**//a design approach where wireless signals can be employed as a parameter in shaping of spaces//**

read more on the theoretical part of this research on the thesis blog [[http://emperors.kucjica.org|emperors.kucjica.org]]

====== Connect or Not ======

===== Connect or Not, IST Lisbon =====

[[connect-or-not-lisbon|Details on hardware and software development]]

[[connect-or-not-lisbon|{{http://emperors.kucjica.org/wp-content/uploads/2014/09/frontview-all-on-web-1024x625.jpg?300}}]]
===== Connect or Not, K4, Ljubljana =====

[[connect-or-not-ljubljana|Details on hardware and software development]]

[[connect-or-not-ljubljana|{{http://emperors.kucjica.org/wp-content/uploads/2014/09/ida-web.jpg?300}}]]

===== Connect or Not, Buhne A, Zurich =====

[[connect-or-not-zurich|Details on hardware and software development]]

[[connect-or-not-zurich|{{:image1772.png?nolink&300|}}]]
====== Quadricone ======

===== Les Urbaines installation in Espace Arlaud, Lausanne =====

[[quadricone-installation|Details on hardware and software development]]

[[quadricone-installation|{{:arlaud-sunday-allpoints.jpg?nolink&300|}}]]

===== Stage Digital I scaled model, Buhne A, Zurich =====

[[quadricone-model|Details on hardware and software development]]

[[quadricone-model|{{http://emperors.kucjica.org/wp-content/uploads/2015/04/qua0-1024x672.jpg?nolink&300|}}]]


====== RKNFG ======
RKNFG installation was developed during a three week residency in Atelierhaus Salzamt. It was shown at the consecutive exhibition [[http://www.linz.at/images/Expand_Explore_Expose.pdf?_blank|Expand, Explore Expose in the Salzamt gallery]], September 2012

[[RKNFG|Details on hardware, software and development of RKNFG]]

[[RKNFG|{{:rknfg-exhibition-0.jpg?nolink&300|}}]]

---

====== COMPONENTS: ======

====== connect or not application ======

[[http://connectornot.emperors.kucjica.org]]

@ playstore: [[https://play.google.com/store/apps/details?id=net.lmag.connectornot]]

@ github: [[https://github.com/LouisMagarshack/connectornot]]

[[android_ips|indoor positioning]]

====== data visualisation ======

d3 js

[[workflow]]

[[http://kucjica.kucjica.org/emperors-vizi/|results]]
====== lisboa: connect or not at IST Alameda campus ======

Beagle Bone Black + 2 Arduinos + 4 RGBW high power LEDs + 4 servo motors + 7m stretchable fabric + Android app + people

[[lisboa-worklog|worklog]]
====== LED wall @ EPFL (apr/may 2014) ======

  * [[http://www.adafruit.com/products/306|LPD8806 LED strip]] [[https://solarbotics.com/download.php?file=1889|(datasheet)]] 
  * [[http://arduino.cc/en/Guide/ArduinoYun|arduino yun]] [[https://github.com/adafruit/LPD8806|(LPD8806 arduino library)]] / raspberry pi [[https://github.com/adammhaile/RPi-LPD8806|(LPD8806 python raspberry library)]] 
  * [[http://connectornot.emperors.kucjica.org/|connectornot]]


 [[worklog-at-bc|worklog]]

{{:img_20140526_210240.jpg?nolink&400|}}

----

====== MMC K6/4 residency (jan/feb 2014) ======

[[worklog]]


----

====== Workshop @ MMC Zavod 6/4, January 2014 ======

[[mmc-workshop-outcomes|workshop outcomes]]

how to interface wirelessness with space? use light, sound and movement to render experiencable the events that take place in the field of high frequency electromagnetic radiation


{{::workshop-scheme.png?590}}

===== what are wireless signals? =====

{{http://imgs.xkcd.com/comics/electromagnetic_spectrum.png?590}}

===== how can we perceive them? =====

  * computer (or other) wireless card (wifi only)

  * mobile phone, smart phone, tablet (umts, gst, wifi)

===== how can we explore / exploit them? ===== 

[[artworks|some artworks]]


===== workflow =====


  - install //connect or not// application
  - receive data in pd
  - send values to arduino
  - control lights, sound, motors


==== receive OSC data ====

  * start pd-extended
  * create the basic receiving objects

[[pd-osc-receive|basic receiving pd patch]]

  * run //connect or not// on your phone, press START
  * observe the data you receive
  * create the structure to receive each information separately (use OSC addresses)

[[pd-osc-receive-further|receiving pd patch further]]
==== arduino controls: LED test ====

  * connect your arduino. 
  * start the arduino IDE
  * under Tools > choose the correct Board (eg. Arduino Uno) and Port (eg. /dev/ttyACM0)
  * paste the following code in your sketch:

[[LED_serialread_arduino]]

  * start pd-extended
  * paste the following code in a text editor and save with .pd extension

[[LED_serialread_pd]]

  * open the file in pd
  * connect the LED to pin 8 (longer leg) and ground (shorter leg)
  * change the values on the slider and observe the LED switching on and off; use messages 0 and 255 to do the same


==== arduino controls: sound ====

  * connect the arduino and start the IDE (follow the steps from the LED test)
  * connect the speaker to pin 8 (red wire) and ground (black wire)
  * paste the following code in your sketch and upload it to the board

[[SND_serialread_arduino]]

  * paste the pd code into a text file, save with a .pd extension and open in pd

[[SND_control_pd]]

  * change the value of the green slider (pitch)


==== arduino controls: motors ====

  * connect the arduino and start the IDE (follow the steps from the LED test)
  * paste the following code in your IDE sketch and upload it to the board

[[MOVE_serialread_arduino]]

  * connect the data (usually yellow) wire of the motor to pin 11. connect power (red wire) and ground (black wire) to power (5v) and ground of arduino
  * paste the following code in your favourite text editor and save with .pd extension; then open the file with pd-extended

[[MOVE_control_pd]]


==== finally, connect the OSC data stream to a physical output ====

  * open the [[pd-osc-receive-further|receiving pd patch further]] 
  * open the pd control patch (light, sound, movement)
  * copy the objects and paste to the first patch
  * assign received values to different reactions of the physical system (blinking of light, speed of movement, sound pitch and duration...); fiddle with received values until you reach desired interaction;


===== results =====

here the result

{{::wireless-space-pd.png}}

and here [[the patch]]


----

====== technical development: general ======


===== step 1: scanning wifi =====

==== airodump-ng ====

  * airodump-ng: scans the surrounding APs; returns details on ESSID, encryption type, power level, packets going through... writes the dump to a file, able to write .cap and .csv files;
  * has to be run as sudo

<code>
/etc/init.d/network-manager stop
killall -9 nm-applet
airmon-ng start [iface] 
airodump-ng [monitor iface]

iwlist

  • iwlist scan returns a list of access points with their ESSID, quality, frequency, power… and can write to a file, for example like this
iwlist [iface] scanning

scripting wifi scan with python

problems:

  • output of aircrack scan cannot be passed further to python directly
  • need data on all available access points in the range of my wifi card
  • need data on network activity (e.g. how many devices are connected to it, ideally what is the data flow)
  • most network tools scan traffic within a specific network, but do not give an overview at the same time

solutions:

wardrivintg.py

a python script that calls in the results of iwlist scanand returns a list of ESSIDs

wardriving.py

scapy

pcapy

step 2: controling the motors

arduino servo

try loop movement with arduino servo 180 test

interactive positioning using python serial arduino servo serial interactive

python + arduino API

python + arduino, 4 servo control

step 3: pass the wifi scan (airodump-ng) data to the motors

using the extended-python-arduino-prototyping-api i call [motor].write(value) with a value determined from the scan

for example, with a running instance of airodump-ng which scans the traffic on a particular network ( –bssid 14:D6:4D:B6:97:3C) and writing to a file (-w /home/selena/doc/synergia/sator/scanning/salzamt1109)

airodump-ng --bssid 14:D6:4D:B6:97:3C -w /home/selena/doc/synergia/sator/scanning/salzamt1109 mon0

we can read the amount of data packets that have passed through the network with a python function like this:

def lookup(table):
    scan=open(table)
    next(scan)
    scanDict=csv.DictReader(scan, delimiter=',')
    for adict in scanDict:
        if adict.has_key(' # IV'):
            data=adict.get(' # IV')
            print data


airodump='/home/selena/doc/synergia/sator/scanning/salzamt1109-01.csv'

when we call the lookup function, it returns the value of the '# IV' key in a dictionary that is contained in the DictReader object 'scanDict'. the returned value is a string, so we have to turn it into an integer.

packets=int(lookup(airodump))

the full script is available here


other

start.1508324356.txt.gz · Last modified: 2017/10/18 10:59 by zoza