September 2008 - Posts

Google Virus !
Wednesday, September 17, 2008 6:25 AM

האמת היא שלא חשבתי שאמצא את עצמי כותב פוסט כזה. או יותר נכון לא חשבתי ככה עד לפני כמה חודשים. מאז קרו כמה דברים והדבר האחרון שקרה הוא התקנה של Google Desktop. מה שהתחיל בתור אפליקציה נחמדה שמחזיקה Indexing של כל הקבצים הפך למשהו אחר לחלוטין.

בשבוע האחרון ביצעתי porting של אפליקציה גדולה מ- VC6 אל VS 2008. בקוד יש חלקים שעובדים עם OpenGL ומסתבר שהספריות שהיו קיימות ב- VS 2005 (תחת התיקיה include\gl) נעלמו ואינן. במקומן קיבלנו את Windows SDK שמגיע כהתקנה נפרדת ושם כבר קבצים בשמות אחרים. בעודי מחפש כל מיני lib-ים ו-h-ים חשבתי לעצמי שיהיה הרבה יותר פשוט להתקין את Google Desktop Search על מנת למצא את הקבצים שמפוזרים בין Program Files לבין Windows. איזו טעות זו היתה!

כבר בשלב ההורדה התחלנו ברגל שמאל, גוגל ואני. אמנם נכנסתי אל google.com אבל החברה שעבורה אני מבצע את העבודה מחוברתאל האינטרנט דרך אירופה וקיבלתי גרסה בשפה מאד מעניינת (בלגית או משהו דומה). כמובן שבזמן ההתקנה ביטלתי את כל ה- V-ים האפשריים אבל למרבה ההפתעה בכל זאת זכיתי להרבה מאד דברים מעניינים מותקנים בהרבה מאד מקומות, כולל ב- Outlook וכל מיני כלים מוזרים ומשונים. מעניין שגוגל יוצרים browser עם מראה פשוט ונקי ובאותו הזמן דוחפים תוספות לכל מוצר אפשרי של מיקרוסופט :-)

מהר מאד אני מוצא את עצמי עם שלל התקנות שלא ביקשתי ועם google desktop בשפה שאני לא מבין. החוויה כולה היתה כל כך שלילית שהחלטתי לוותר ולהסיר את כל החבילה כולה. מסתבר שהחוויה של ההסרה גרועה בהרבה. דיי מזכיר לי כל מיני חברות ישראליות שמספקות שירות סוג ד', כאלה שמזדרזות לחבר אותך בלי הרבה שאלות לשירות חדש אבל מתישות עד כמה שניתן כאשר אתה מבקש להתנתק. מסתבר שחברינו בגוגל מאמינים גם הם בשיטה הזו.

ניסיתי להפעיל את קובץ ההתקנה המקורי (הפעם באנגלית) אבל הוא מוכן רק לשדרג ואין שם בכלל אפשרות להסרה. ניסיתי לחפש בתפריט התחלה אפשרות להסרה, כמובן שאין. בשלב הזה הבנתי כבר שהסרה היא לא משהו שמישהו בגוגל דמיין לעצמו כשימושי... לוח בקרה, הסרה והוספה של תוכניות, ושם יושב לו google desktop. כמה נפלא חשבתי לעצמי. לחצתי על הסר (כמובן הכל באנגלית) והתחלתי לחכות. חיכיתי עוד קצת. תיקנתי שני באגים. כוס קפה... ה- process רץ ברקע ולוח הבקרה תקוע. עוד נסיון, ועוד אחד, restart, ועוד נסיון. כלום. מישהו בגוגל לא טרח אפילו לבדוק את האופציה להסרה.

אבל שימו לב, זה לא הכל. בצבא לימדו אותי שמה שלא נפתח בעזרת מברג נפתח בעזרת כת של גליל (לא לנסות עם M16 !!). פתחתי בחוסר סבלנות Task Manager, מצאתי מהר מאד את Google Desktop והרגתי אותו בלי לחשוב פעמיים. מחשב זמני, אז לא אכפת לי מה- registry, ניגשתי לתיקיה של גוגל תחת program files ומחקתי את כולה כמו שהיא. ... או בעצם לא... מסתבר שהתיקיה נעולה. חשבתי לעצמי שבטח מדובר באחת התוספות שלא ביקשתי אז סגרתי את outlook ואת explorer, ועדיין כלום. צעד הבא הוא כמובן להתחיל להרוג כל מיני תהליכים כאלה כמו עותקים של explorer, acrobat וכד'. שוב אני מוצא את עצמי עם Task Manager ונחשו את מי אני פוגש שם. את Google Desktop, והבחור חי ונושם. עוד יריה על End Process. ושוב מחיקה של התיקיה לא עובדת. עכשיו זה מתחיל להשמע מוזר. גם כלי יעיל בשם unlocker שמשחרר נעילה של קבצים נכשל מלשחרר את התיקיה.

אחסוך לכם את כל התהליך ואגיע ישר למסקנות: התוכנה התמימה הזו google desktop מתקינה את עצמה עם עוד כל מיני תוספות קטנות שלא ביקשנו. יותר גרוע מכך יש שני עותקים של ה- process שרצים ברקע. בכל פעם שהורגים אחד מהם השני - שימו לב טוב לזה - יוצר עותק נוסף בעזרת RegSvr32. בכל פעם שהורגים אחד מהם השני יוצר תיקיה חדשה, מעתיק אליה קבצים ומפעיל אותם בעזרת RegSvr32, בתהליך שבסופו נוצר עותק מקביל. לא משנה את מי מהם הורגים השני ידאג להחזיר את המצב לקדמותו.

אני חייב להודות שמדובר בוירוס שלקח לי הכי הרבה זמן להסיר (מעל לשעה). שום תוכנת ריגול, וירוס, ואפילו אנטי וירוס לא נדבקים למערכת בצורה כל כך טובה מבלי להשתמש בדרייבר. כל הכבוד לגוגל על ההישג הטכנולוגי המרשים. יותר אני לא מתקין מוצרים של גוגל על תחנות עבודה ואם יש לי ברירה אז בכלל!

 

אסף

by AsafShelly | 4 comment(s)

תגים:OFFTOPIC

Windows NT Kernel as a well designed parallel system
Monday, September 15, 2008 5:59 PM

Hi all,

Thought you might find this interesting:

http://softwareblogs.intel.com/2008/09/15/is-dos-the-ideal-parallel-environment-part-iii/

Comments are most welcome (Gadi... :)

 

Asaf

by AsafShelly | 3 comment(s)

תגים:Parallel Computing, multiprocessing, multicore, http://AsyncOp.com, DEV

How to make a good Power Point presentation video
Sunday, September 14, 2008 5:19 PM

Watch learn and enjoy.

http://www.presentationzen.com/presentationzen/2007/04/powerpoint_some.html

by AsafShelly | 2 comment(s)

תגים:OFFTOPIC

Super Smart Pointer - Sample Code
Thursday, September 11, 2008 10:14 AM

In continuance to my last post about memory allocations from mapped files http://blogs.microsoft.co.il/blogs/asafshelly/archive/2008/09/07/super-smart-pointer.aspx (and because Lior asked for it :) Here is the sample that I used to test the timing:

Includes: 

#include "stdafx.h"
#include <WINDOWS.H>
#include <conio.h>

#define PAUSE { puts("Ready"); getch(); }  

 

Interface:

class AllocBuff
{
public:
   virtual bool New(int bytes) { return(NULL); }
   virtual bool Delete() { return(false); }
   virtual char* Lock() { return(false); }
   virtual bool Unlock() { return(false); }
};   

 

Memory Implementation

class MemBuff: public AllocBuff
{
   char* buff;
   CRITICAL_SECTION cSection;
public:
   virtual bool New(int bytes)
   {
      InitializeCriticalSection(&cSection);
      try { buff = new char[bytes]; } catch(...) { buff = NULL; }
      return(buff);
   }
   virtual bool Delete()
   {
      DeleteCriticalSection(&cSection);
      delete[] buff; buff = NULL;
      return(true);
   }
   virtual char* Lock()
   {
      if (!buff) return(NULL);
      EnterCriticalSection(&cSection);
      return(buff);
   }
   virtual bool Unlock()
   {
      LeaveCriticalSection(&cSection);
      return(true);
   }
};   

 

File Mapping Implementation 

class FileBuff: public AllocBuff
{
   static HANDLE hFile;
   static HANDLE hFileMap;
   static char* nextBuff;
   static int tempID;
   static int pageSize;   int size;
   char* buff;
   char* mappedBuff;
   CRITICAL_SECTION cSection;
public:
   static bool Init()
   {
      SYSTEM_INFO info;
      GetSystemInfo(&info);
      pageSize = info.dwAllocationGranularity;
      char fileName[MAX_PATH];
      GetTempFileName(".", "FileBuff", tempID, fileName);
      tempID++;
      if (INVALID_HANDLE_VALUE != (hFile = CreateFile(fileName, GENERIC_READ|GENERIC_WRITE, FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY|FILE_FLAG_DELETE_ON_CLOSE, NULL)))
      {
         if (NULL != (hFileMap = CreateFileMapping(hFile, NULL, PAGE_READWRITE, 4, -1, NULL))) return(true);
      }
      return(false);
   }
   virtual bool New(int bytes)
   {
      if (NULL == hFileMap) { if (!Init()) return(NULL); }
      InitializeCriticalSection(&cSection);
      size = bytes;
      buff = nextBuff;
      nextBuff += bytes;
      mappedBuff = NULL;
   }
   virtual bool DeInit()
   {
      // .... etc.
      return(CloseHandle(hFile));
   }
   virtual bool Delete()
   {
      size = 0;
      buff = NULL;
      DeleteCriticalSection(&cSection);
      return(true);
   }
   virtual char* Lock()
   {
      EnterCriticalSection(&cSection);
      if (!(mappedBuff = (char*)MapViewOfFile(hFileMap, FILE_MAP_ALL_ACCESS, 0, ((DWORD)buff)/pageSize*pageSize, ((DWORD)buff) + size - ((DWORD)buff)/pageSize*pageSize)))
      {
         DWORD dw = GetLastError();
         LeaveCriticalSection(&cSection);
         return(NULL);
      }
      mappedBuff = mappedBuff + ((DWORD)buff)%pageSize;
      return(mappedBuff);
   }
   bool Unlock()
   {
      if (mappedBuff)
      {
         UnmapViewOfFile(mappedBuff);
      }
      LeaveCriticalSection(&cSection);
      return(true);
   }
}; 

 

int FileBuff::tempID = 1;
int FileBuff::pageSize = 0;
char* FileBuff::nextBuff = (char*)0x10000;
HANDLE FileBuff::hFile = INVALID_HANDLE_VALUE;
HANDLE FileBuff::hFileMap = NULL;       

 

Test Application - main

int _tmain(int argc, _TCHAR* argv[])
{

   char* str = "In continuance to my last post about memory allocations from mapped files http://blogs.microsoft.co.il/blogs/asafshelly/archive/2008/09/07/super-smart-pointer.aspx (and because Lior asked for it :) Here is the sample that I used to test the timing...\n";

   OutputDebugString("Starting\n");
   for (int i = 0; i < 2000000; i++)
   {
      // AllocBuff* buff = (AllocBuff*)new MemBuff;
      // AllocBuff* buff = (AllocBuff*)new FileBuff;
      buff->New(255);
         char* ptr = buff->Lock();
         if (!ptr) throw("char* ptr = buff->Lock()");
            memcpy(ptr, str, strlen(str) + 1);         buff->Unlock(); ptr = NULL;
      buff->Delete();
      delete buff; buff = NULL;
   }
   OutputDebugString("Done\n");

   PAUSE;

   return 0;
}  

Use DebugView to mesure times.

Asaf

 

by AsafShelly | with no comments

תגים:Parallel Computing, multiprocessing, multicore, http://AsyncOp.com, DEV, OFFTOPIC

Super Smart Pointer
Sunday, September 07, 2008 4:17 PM

Memory management is one of the key elements in the world of software. So many bugs come from misusing memory pointers, over-writing buffers, racing over buffers (several threads use the same buffer), deleting a buffer while it is still used, running out of memory, and so many more. To solve this problems we have seen so many methods and technologies used starting with Locks which are flags that help manage access to a buffer, Garbage Collector that helps manage buffer deletion from the Heap, the user of a Stack, Smart Pointers, Reference Counting, Page Protection flags, and so many other techniques and technologies. The focus of this article is memory management for the parallel world. When a buffer is allocated it has a single owner and only one thread that has a pointer to it. This thread, as the only owner and the only one who knows the buffer, is also the only one that deallocate it. We say that the pointer to the buffer is also the buffer's ID. If you know the pointer then you can use the buffer. When multiple threads are used a synchronization method is usually required. Locks are used as flags that manage access to buffers. Only a single thread owns the lock and only the owner of the lock should access the buffer. However any thread that knows the buffer can, at any time, access the buffer and potentially damage the operation of the buffer that got a successful lock and was assumed to be protected. Memory deallocation is a big problem in both parallel and serial world. One of the preferred models that improve manageability is Auto-Pointers or Smart-Pointers. Such an object will hold on to the buffer for as long as there are users to it and deallocate it when it is no longer in use. Such as object can count the users and also manage locks. A user is a part of the code that is holding on to the buffer and needs to know the buffer. There can be multiple users for a single thread, or several threads in a pool can execute several parts that are using the thread. Buffer users are not related to the number of threads in the system and are more dependant on the number of objects and functions (mechanisms) that need to know the buffer. Between all the users there can only be a single owner. This owner is the only one that is allowed to work with the buffer. Most implementations of Smart Pointers are designed to help manage the users and allow appropriate buffer cleanup and do not really consider multiple threads using the objects. Around the time multicore CPUs are introduced we also have 64 bit CPUs. These allow the use of more than 2GB or application memory (3GB with global system configuration). The problem today is that most applications were not designed for 64' CPUs and software companies hesitate from the move to 64 bit. With that we also know that the drivers available for 64 bit systems do not cover a wide verity of devices as available for the 32 bit version of the operating system.The problem that many software companies are facing today is that the 3GB limit is almost reached. An even bigger problem is that during normal system operation buffers are allocated and deallocated. This means that eventually the memory is highly fragmented and an application that uses 1.2GB for its normal operation consumes 2GB of memory for normal operation just because it ran for long enough. The solution to these problems is a Super Smart Pointer that supports:

1. Internal reference count of buffer users
2. Locking support associated to the buffer
3. Automatic memory management that will move buffers in memory to allow memory defragmentation and allow more than 4GB of memory address space in a 32 bit application.

 The basic object interface (to be wrapped with operators) should support:User Interface:

1. Allocate Buffer
2. Duplicate Use (Add Reference)
3. Stop Use (Release, and last release will delete)

 Owner Interface:

1. Lock Buffer
2. Unlock Buffer
3. Test Lock

 Heap Operations:

1. Create Heap
2. Destroy Heap
3. Lock / Unlock Heap
4. Crash Dump

 The technology that we will use for our Heap is not a memory heap. Instead we will use a file. Files are not limited in size of 4GB and are relatively easy to use. The system also supports advanced access management to files and file sections though the File System as an Object Store. The Heap operation Create Heap with therefore be CreateFile. The file is of course created with the temporary flag so that the system tries not to write to the physical file. This way the system will only write to the physical file when we run out of available RAM.Equivalently we use CloseHandle in the Heap operation Destroy Heap. We can also use the creation flag of CreateFile that tells the system to destroy the file automatically when the last handle is closed.We could use a file for every allocated buffer but we won't because it increases complexity and damages runtime. The Heap operations of Lock and Unlock for the Heap are used to block access to all uses and completely prevent the Heap from being modified (or read). This is useful in the case that a single thread detected a serious bug. We want to freeze the system as soon as possible to save the memory state. Here comes the last Heap operation called Crash Dump. This is fairly simple: close the file without deleting it. If the file was marked for automatic deletion then copy it entirely to a crash-dump file that can be later investigated. User operation of Buffer Allocation is as simple as saving the place in the file. It is enough to find the next available location and return this location as the pointer to the buffer (as buffer ID). Buffer deletion is as simple as marking the location as empty.Every buffer starts with the size of the buffer and with a reference count. The pointer returned is to the data past these two values so for example a pointer with the value of 0x1208 has reference count at address 0x1204 and size at 0x1200. User operation of Duplicate Use is simply incrementing the reference counter. Operation of Stop Use is decreasing the counter and when the counter is at zero mark the object as deleted (see existing memory managers for realworld implementations). Last but not least of this implementation is the use of Memory Mapped Files.The Owner operation of Lock Buffer will lock an internal object and only if the lock is successful a pointer is returned. If the lock fails then the user is not the owner and is not allowed to write to the buffer or read from it.When the pointer to the buffer is returned only when we perform the operation Lock then until that point we don't really care where the buffer is in real memory or even if it is in real memory at all. We use the Memory Mapped Files API MapViewOfFile to map the buffer's data into the process's address space. We lock the buffer ownership and we lock the buffer into memory. At this point it is safe to use the buffer. When we are done we use the Unlock Buffer operation that will unlock the buffer from memory and unlock the buffer ownership. Unlocking the buffer from memory will also allow virtual memory defragmentation. By using memory mapped files we can also use the same application design with multiple processes instead of multiple threads, which would increase application stability because threads can damage each other's memory but processes cannot touch local memory that belongs to another process. A thread can accidentally erase the stack that belongs to another thread and thus damage local scope variables. A process that did not map a memory buffer because it is not the owner cannot damage this buffer. Basically this means that unless there are really exceptional reasons any buffer is mapped to a single process at a time. Files are not limited to 4GB, we get automatic memory buffer swapping mechanism that will defrag the virtual memory for us, and it is very easy to get a crash-dump file that can be analyzed after the application crashed.              

 

by AsafShelly | 1 comment(s)

תגים:Parallel Computing, multiprocessing, multicore, http://AsyncOp.com, DEV

Google - what happened??
Thursday, September 04, 2008 12:59 PM

In continuation to this post by Eran Kampf: http://www.developerzen.com/2008/09/02/google-chrome-is-google-replacing-my-os-again/

I originally thought about a simple reply but it came out so long that I ended up making this a formal post.

Long long time ago people came up with the idea of working remotely without any hard drive on the machine.
Since then came AJAX because people did not like the user experience of browsers, and online content failed almost completely. People did not want to pay for a song that is "somewhere out there", for the right to see a movie but not own it, and for working on their documents (some times sensitive) without receiving their own copy.
Google woke up one day and decided to do that... for some reason. It's probably a nice browser.

Originally movie producers thought that DVDs would take away all revenue. Today the movie is just a promotion for the DVD. The reason is that people feel better when they own the things that they like. When you put effort in a document you like it. Ask any developer that wrote a feature that needs to be taken out of the application.

The two basic rules I have for you are:
1. It takes at least two dots to make a line, and
2. Never argue with the way customers feel about things.

 

I had another interesting experience related to this: a while back I noticed that MSDN searches suggest common keywords and thought to myself: that it is an amazing feature, that it is about time that the online version did what the offline version does, and that this has to be pretty heavy to download... A few days ago I noticed the same feature on Google search... "Hmm..." I though to myself, and was wondering how heavy download was this and were do they keep this information hidden (not a big issue, it's just not interesting enough to open a tool for it).

Funny how the wheel turns...

 

Asaf

 

by AsafShelly | 6 comment(s)

תגים:OFFTOPIC